Blood oxygen saturation degree image contrast learning method based on multi-scene video

Abstract

The invention discloses an oxyhemoglobin saturation image contrast learning method based on a multi-scene video, and belongs to the field of video image processing. According to the method, a multi-wavelength characteristic representation module is introduced into the VBOSNet, and spectrum prior information is fused into the model to enhance the discrimination capability of SpO2 related characteristics. In order to improve the generalization performance under the condition of limited data and cross-individual difference, the VBOSNet is combined with a physiological feature extractor based on comparative learning, and a targeted comparative learning sample mining strategy is designed, so that more stable and invariant signal representation is realized. In addition, a self-built ICU SpO2 data set is introduced, and the data set is collected from a real clinical environment and used for supporting algorithm development and performance evaluation under complex conditions.

Description

Technical Field

[0001] This invention relates to the field of video image processing, and more specifically to a blood oxygen saturation image comparison learning method based on multi-scene videos. Background Technology

[0002] Blood oxygen saturation (SpO2) is a crucial parameter for assessing cardiovascular and respiratory function. Continuous monitoring of SpO2 can promptly detect rapid SpO2 desaturation caused by respiratory diseases such as asthma and chronic obstructive pulmonary disease (COPD), thereby reducing the risk of hypoxemia-related complications and potentially fatal consequences. Current SpO2 measurement methods are divided into invasive and non-invasive categories. While arterial blood gas (ABG) analysis is highly accurate, it is an invasive method and not suitable for real-time monitoring. Pulse oximeters, on the other hand, offer a non-invasive, real-time alternative and are widely used in clinical practice. However, their contact-based design limits the flexibility of use and may affect comfort and measurement accuracy with prolonged wear. Therefore, video-based SpO2 estimation methods are gradually gaining increasing attention due to their non-invasiveness, low cost, and long-term sustainability.

[0003] Currently, video-based SpO2 estimation methods can be broadly categorized into two types: traditional signal processing methods and deep learning methods. Signal processing techniques typically rely on ratio analysis across spectral channels. Existing technologies use RGB cameras to measure at a distance of 1.5 meters and achieve reliable SpO2 estimation for healthy subjects through signal-to-noise ratio enhancement, ROI optimization, and red / blue channel ratio calculation. In contrast, deep learning methods can improve scene generalization capabilities by leveraging diverse training data. One existing smartphone-based solution utilizes fingertip occlusion combined with a CNN model, validating its feasibility on induced hypoxemia data. Converting video sequences into spatiotemporal maps and combining them with the AC / DC feature fusion method of CNNs effectively improves estimation accuracy. More recently, the CCSpO2Net solution combines an MLP-Mixer module with adaptive skin segmentation, achieving significant improvements over previous non-contact methods in controlled environments.

[0004] While the aforementioned studies achieved satisfactory results in controlled laboratory environments, they still face significant performance degradation in cross-individual and real-world scenarios due to environmental interference, individual variability, and limited spectral information. Specifically, skin color differences between individuals lead to variations in facial signal patterns, thus weakening the reliability of cross-subject modeling. Complex real-world environments (such as ICUs) introduce substantial noise due to varying lighting conditions and non-standardized camera perspectives. Furthermore, most current methods rely solely on the RGB three channels, failing to fully utilize richer spectral information to improve estimation accuracy. Summary of the Invention

[0005] To address the aforementioned shortcomings in the existing technology, this invention provides a blood oxygen saturation image comparison learning method based on multi-scene videos.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: A blood oxygen saturation image contrast learning method based on multi-scene videos includes the following steps: S1. Preprocess the input video to extract physiological signal-related regions and generate a spatiotemporal map; S2. Constructing multi-wavelength feature representation based on the light absorption characteristics of hemoglobin; S3. Construct training sample pairs using a contrastive learning strategy, including anchor samples, positive samples, and negative samples; S4. Train the neural network model by combining the multi-wavelength features and contrastive learning; S5. Estimate blood oxygen saturation based on the trained model.

[0007] Furthermore, step S1 includes: S11. Use detection algorithms to extract facial or fingertip areas; S12. Perform spatial downsampling and temporal window sliding to generate a spatiotemporal map.

[0008] Furthermore, step S2 includes: S21. Select a target wavelength relevant to the absorption characteristics of hemoglobin; S22. Calculate the Euclidean distance between the pixel RGB value and the wavelength reference value. The calculation formula is as follows:

[0009] In the formula, This represents the calculated pixel value. , and These are the reference RGB values ​​for the target wavelength, and , and These are the actual RGB values ​​of the pixels in the image; S23. Generate a multi-channel feature map based on the calculated Euclidean distance.

[0010] Furthermore, step S3 includes: S31. Construct positive and negative sample pairs based on individual identity and blood oxygen saturation level, where positive samples come from different individuals but have the same blood oxygen saturation, and negative samples come from the same individual but have different blood oxygen saturation. S32. Perform data augmentation on anchor point samples to generate augmented positive samples. Augmentation operations include spatial augmentation, temporal augmentation, and color augmentation.

[0011] Furthermore, step S4 includes: S41. Construct a neural network model, the model including a physiological feature extraction module, a multi-wavelength feature extraction module, and a feature fusion module; S42. Configure training parameters, including using the Adam optimizer and setting the learning rate. The weight decays to The batch size is 16, and an early stopping strategy is adopted. Training is performed for a maximum of 100 epochs. If the validation set shows no improvement for 30 consecutive epochs, the training is stopped early. S43. Define a composite loss function for model optimization, wherein the loss function includes mean squared error loss. Negative Pearson correlation coefficient loss Extended triplet loss Loss compared with multiple positive samples ; S44. Perform model training, using the composite loss function for backpropagation and parameter updates, where the overall loss is calculated by weighted summation:

[0012] in, To balance the hyperparameters, which are used to adjust the contribution of each component in the joint optimization.

[0013] Furthermore, in S41, the physiological feature extraction module uses an improved ResNet structure combined with a Squeeze-and-Excitation attention mechanism, the expression of which is:

[0014] In the formula, Conv represents the convolution operation, and SE represents the attention mechanism; The multi-wavelength feature extraction module employs a multi-scale channel attention mechanism, the expression of which is:

[0015] In the formula, LAtt represents local channel attention, GAtt represents global channel attention, and σ is the sigmoid function. This indicates element-wise multiplication at the channel level; The feature fusion module stitches together physiological features and multi-wavelength features in the channel dimension.

[0016] Furthermore, the extended triplet loss in S43 Represented as:

[0017] In the formula, Indicates anchor point sample, To enhance positive samples, These are positive and negative samples selected from the dataset using OCS, respectively. Let be the distance function. To control the boundary hyperparameters for the separation between positive and negative samples; Multi-positive sample contrast loss Represented as:

[0018] In the formula, Indicates anchor point sample, For its two corresponding positive samples, For negative samples, Represents the computation of vectors cosine similarity, Temperature parameters for controlling the steepness of the distribution, Indicates batch size.

[0019] Furthermore, in step S5, a Transformer-based regressor is used to predict blood oxygen saturation values, the expression of which is:

[0020] In the formula, FC is the fully connected layer, DeiT is the Transformer architecture, and Neck is the feature fusion module.

[0021] The present invention has the following beneficial effects: This method combines multi-wavelength feature representation with contrastive learning to enhance spectral sensitivity and improve cross-individual generalization. VBOSNet integrates a contrastive learning-enhanced physiological feature extractor, a spectrally-aware multi-wavelength feature extraction module, and a lightweight Transformer-based regressor. These components work synergistically to effectively capture SpO2-related features and ensure robustness against individual differences and environmental noise. Extensive experimental results on the PURE, FBO, and self-built ICU datasets demonstrate that VBOSNet significantly outperforms existing methods, achieving substantial reductions in MAE and RMSE even in complex clinical settings. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the blood oxygen saturation image comparison learning method based on multi-scene video according to the present invention.

[0023] Figure 2The diagram below is a schematic of the framework design of an embodiment of the present invention, wherein (a) is the data preprocessing process including target region extraction, spatial downsampling and spatiotemporal map generation, (b) is the multi-band spectral feature embedding method (MWL) based on Euclidean distance filtering, and (c) is the VBOSNet model architecture.

[0024] Figure 3 This is a schematic diagram of the wavelength selection principle for SpO2 estimation in the visible light band according to an embodiment of the present invention, wherein (a) is the molar extinction coefficient curve of hemoglobin at different wavelengths; and (b) is a comparison of the absorbance capabilities of HbO2 and Hb in different wavelength ranges.

[0025] Figure 4 This is a schematic diagram illustrating the comparative learning principle of an embodiment of the present invention.

[0026] Figure 5 This is the overall architecture of VBOSNet in this embodiment of the invention. (a) is the Physiological Feature Extraction (PFE) module; (b) is the Multi-Wavelength Spectral Feature Extraction (MWFE) module; (c) is the Channel Fusion Module (Neck); and (d) is the Deit-based SpO2 estimator.

[0027] Figure 6 This document presents a consistency analysis and Bland-Altman plot for an embodiment of the present invention. The first row compares the predicted values ​​with the actual values, and the second row displays the corresponding Bland-Altman plot. Detailed Implementation

[0028] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0029] A blood oxygen saturation image contrast learning method based on multi-scene videos, such as Figure 1 , 2 As shown, it includes the following steps: S1. Preprocess the input video to extract physiological signal-related regions and generate a spatiotemporal map; S2. Constructing multi-wavelength feature representation based on the light absorption characteristics of hemoglobin; like Figure 3As shown, traditional pulse oximeters use dual-wavelength light sources, typically selecting wavelengths of approximately 630 nm and 900 nm. This selection is based on the differential absorption characteristics of oxyhemoglobin (HbO2) and deoxyhemoglobin (Hb) in these spectral bands. Consumer-grade RGB cameras are limited to the visible light range of approximately 380–780 nm, and therefore cannot directly capture information in the infrared band. To simulate the Hb and HbO2 absorption contrast commonly used in infrared-based measurements, this study aims to extract similar discriminative features from the visible spectrum. Based on Prahl's research, which calculated the absorption curve of hemoglobin using the molar extinction coefficient (as shown in Fig. 3(a)), this study introduces an absorption contrast ratio. It is defined as the ratio of the molar extinction coefficients of HbO2 to Hb, thereby quantifying the spectral differences between the two.

[0030]

[0031] in, The molar extinction coefficient of oxyhemoglobin (unit: ), The molar extinction coefficient (units are the same) represents the molar extinction coefficient of deoxyhemoglobin. As shown in Fig. 3 (b), the results are displayed at 900 nm. = 0.364, and similar values ​​are also observed at multiple wavelengths in the visible spectrum. The values ​​are particularly significant at 456 nm and 486 nm. To quantify the spectral similarity between the input image and these selected wavelengths, this study calculated the Euclidean distance between the RGB values ​​of each pixel in the image and the corresponding RGB values ​​of the target wavelength.

[0032]

[0033] in, This represents the calculated pixel value. , and These are the reference RGB values ​​for the target wavelength, and , and This is the actual RGB value of a pixel in the image. This distance is calculated for every pixel in the entire image, generating a feature map where each pixel value represents the spectral difference between the input image and the selected wavelength. The resulting feature map can modulate the original image using the spectral characteristics of the target wavelength. By stitching the feature maps corresponding to each wavelength (456 nm, 486 nm, 630 nm) along the channel dimension, a three-channel multi-wavelength feature map can be obtained. Each channel encodes the optical absorption characteristics of hemoglobin components at a specific wavelength, thereby enhancing the image's ability to capture SpO2-related optical features.

[0034] S3. Construct training sample pairs using a contrastive learning strategy, including anchor samples, positive samples, and negative samples; In this embodiment, the input spatiotemporal graph is used as the anchor sample, and three types of contrast samples are constructed through carefully designed data mining and enhancement strategies: enhanced positive samples. Positive samples and negative samples Among them, enhanced positive samples The data is directly generated from anchor point samples through a series of data augmentation operations, which fall into three main categories: spatial augmentation, temporal augmentation, and color-based augmentation. Spatial augmentation improves the model's robustness to motion artifacts by applying random perturbations to each row of the spatiotemporal graph, disrupting its original spatial consistency. Temporal augmentation enhances the model's adaptability to temporal distortion by randomly cropping time windows to simulate signal changes under different frame rate conditions. Color augmentation includes ±30% random adjustment of brightness and contrast, as well as Gaussian blur, to simulate changes in lighting and noise introduced by the video capture system.

[0035] The goal of these enhancements is to make It can reflect distortions and noise in the real environment, thus helping the model learn features that remain stable even in complex environments. To ensure that the learned features are both robust and physiologically meaningful, especially for SpO2 estimation, this study further proposes a contrastive sampling strategy guided by individual identity and blood oxygen saturation attributes, called the Oxy-Contrast Sampling Strategy (OCS). This strategy aims to guide the model to learn discriminative features that are highly correlated with SpO2 levels, while maintaining the robustness of the features.

[0036] Specifically, such as Figure 4 As shown, given an anchor point sample In this embodiment, positive samples are constructed. These samples, derived from different individuals but possessing the same SpO2 level as the anchor point, were used to construct negative samples. These samples originated from the same individual but differed in SpO2 level from the anchor point. After feature extraction, the representations of the four samples are denoted as follows: In feature space, the smaller the distance between features, the higher the similarity, and vice versa. This embodiment uses... This represents the similarity. The training objective is defined as follows: maximize : Improve the model's robustness to noise and augmentation perturbations; maximize And minimize Enhance the model’s sensitivity to blood oxygenation-related features while mitigating domain bias caused by cross-individual differences such as skin color or facial structure.

[0037] The proposed comparative sample construction method combines robustness to environmental changes with discriminative power of blood oxygen-related features, thus providing an effective feature learning mechanism for accurate and stable SpO2 estimation.

[0038] S4. Train the neural network model by combining the multi-wavelength features and contrastive learning; The overall architecture of the video-based SpO2 measurement network VBOSNet proposed in this embodiment is as follows: Figure 5 As shown, VBOSNet takes a spatiotemporal graph constructed from consecutive video frames as input and contains three functional modules: a physiological feature extraction module (PFE), a multi-wavelength spectral feature extraction module (MWFE), and a DeiT-based SpO2 estimator. PFE extracts dynamic physiological features highly correlated with blood oxygen saturation from the raw spatiotemporal input; MWFE focuses on the spectral components of the multi-channel input, capturing differences in hemoglobin absorption at different wavelengths to enhance VBOSNet's spectral sensitivity to SpO2 changes. Finally, the features extracted by the two modules are fused along the channel dimension (Neck module) and input into a regression network to predict SpO2.

[0039] The physiological feature extraction module (PEE) uses an improved ResNet structure as its basic unit and incorporates a Squeeze-and-Excitation (SE) attention mechanism to enhance channel-level feature modeling capabilities and reduce computational overhead. Figure 5 As shown in section (a), given a size of ( The input spatiotemporal graph is first processed through an initial convolutional layer for channel expansion and basic feature extraction.

[0040]

[0041] in, This represents a sequence of operations consisting of convolution (with a kernel size of k), batch normalization, and the ReLU activation function. Subsequently, the SE module enhances salient features through the following steps: (1) performing global average pooling on the feature map to compress it into channel-level descriptors; (2) modeling the dependencies between channels through fully connected layers and generating channel attention weights; and (3) reweighting the original feature map to highlight task-relevant channels.

[0042]

[0043] in, This represents the sigmoid function. After passing through the Squeeze-and-Excitation (SE) module and being connected to the ReLU activation layer, the final expression of the PFE can be expressed as: .

[0044] Multi-wavelength feature extraction module, such as Figure 5 As shown in section (b), the MWFE module employs a multi-scale channel attention mechanism to adaptively enhance the spectral features associated with SpO2 estimation. First, through a... The convolutional layers map multi-wavelength images to the desired channel space. Subsequently, two parallel attention submodules are constructed. The Local Channel Attention Module utilizes two cascaded... Convolutional layers, combined with batch normalization and ReLU activation functions, capture local channel dependencies while maintaining spatial resolution.

[0045]

[0046] This indicates batch normalization. The Global Channel Attention Module obtains a global representation through adaptive average pooling, and then uses the same structure to model the relationships between global channels.

[0047]

[0048] in, The convolutional weights maintain the same channel dimension as the local branches. The outputs of the two attention branches are summed and then normalized channel attention weights are generated via a sigmoid activation function, which are used to dynamically recalibrate the input features.

[0049]

[0050] This indicates element-wise multiplication. for Activation function, the initial Conv 1×1 Project the number of input channels from 3 to C.

[0051] During the feature fusion stage, the outputs of PFE and MWFE are concatenated along the channel dimension and input into the Neck module to complete feature fusion and dimensionality reduction.

[0052]

[0053] in, The convolutional weights are of size 3. The dimensionality-reduced features are fed into the pre-trained DeiT-tiny as the backbone of the regression network to extract high-level semantic features and decouple SpO2-related components.

[0054]

[0055] This represents a feature concatenation operation along the channel dimension. After passing through the SpO2 Estimator, the features are mapped to a single blood oxygen saturation prediction value via a fully connected layer.

[0056] S5. Estimate blood oxygen saturation based on the trained model.

[0057] Given the multi-module architecture of the proposed network and the introduction of contrastive learning, this embodiment designs a composite loss function framework to simultaneously improve prediction accuracy and generalization ability. In addition to commonly used loss functions such as Mean Squared Error (MSE) and Negative Pearson Correlation Coefficient (NPCC), this embodiment also designs two task-specific loss functions to better suit the needs of this task.

[0058] Extended Triplet Loss: This embodiment designs an improved version of the triplet loss function to better meet the needs of this task.

[0059]

[0060] in, Indicates anchor point sample, To enhance positive samples, These are positive and negative samples selected from the dataset using OCS, respectively. Distance function. Calculate the cosine similarity between two j-eigenvectors. The boundary hyperparameters are used to control the separation between positive and negative samples.

[0061] Multi-Positive Contrastive Loss: Based on the InfoNCE loss framework, this embodiment introduces negative samples and proposes a novel multi-positive contrastive loss, formally expressed as follows:

[0062] In this formula, Indicates anchor point sample, For its two corresponding positive samples, This is a negative sample.

[0063] Calculate vectors cosine similarity, Temperature parameters for controlling the steepness of the distribution, Indicates batch size.

[0064] In summary, MSE ensures the accuracy of numerical regression, NPCC enhances the linear correlation with the true values, extended triplet loss improves the discriminability of features through distance constraints, and multi-positive-sample contrastive loss promotes robust feature clustering. The overall loss is obtained by weighted summation of these features:

[0065] in, To balance the hyperparameters and adjust the contributions of each component in the joint optimization, in this embodiment, their value is set to... .

[0066] VBOSNet was trained on three datasets: the self-built ICU dataset, the PURE dataset, and the FBO dataset. For ICU and PURE, this embodiment used a subject-based partitioning method (8:2 ratio) to avoid leakage of identity information between the training and test sets. For the fingertip dataset containing only six subjects, leave-one-out cross-validation (LOOCV) was used, and the average result of all folds was reported. ROI selection depended on dataset characteristics: ICU used the cheek region (affected by subject position and mask), PURE used the full-face region detected by MTCNN, and FBO directly used the entire frame image because it already covered the region of interest.

[0067] When generating the spatiotemporal graph, a sliding window method is used with a window length of 10 seconds and a step size of 1 second. The optimizer is Adam, and the initial learning rate is set to... The weight decays to The batch size is 16. Training is performed for a maximum of 100 epochs, and the process is stopped early if the validation set shows no improvement for 30 consecutive epochs. All experiments are implemented using the PyTorch framework and run on an NVIDIA RTX-4090D GPU.

[0068] Experimental verification Dataset The PURE dataset was collected in a controlled laboratory environment and contains videos from 10 healthy subjects performing six types of actions: remaining still, speaking, slow / fast translation, and small / medium head rotation. Each action was recorded for 1 minute, resulting in 60 video sequences. Its SpO2 ranges from 89% to 99%, making it suitable for evaluating the model's robustness under mild motion conditions.

[0069] FBO dataset: The FBO dataset was collected in a real hospital environment and includes data from six healthy volunteers aged 20 to 34. Each video segment is between 14 and 19 minutes long. SpO2 values ​​were artificially varied by adjusting the inhaled oxygen concentration to cover a range of 70% to 100%. This dataset reflects real-world home monitoring conditions and provides a robust assessment of the model's generalization ability in hypoxic scenarios.

[0070] Self-built ICU dataset: This dataset, collected from fixed surveillance cameras within hospital wards, realistically reflects the complex clinical environment. The dataset contains 207 video segments (5–10 minutes each) from 20 hospitalized patients, all in a supine position with their faces centered. Heart rate, respiratory rate, and SpO2 values ​​were simultaneously monitored using clinical-grade monitors during recording. Data acquisition was approved by the institution's ethics committee; the approval number will be released upon paper acceptance.

[0071] Experimental setup To evaluate the performance of VBOSNet, this embodiment employs three commonly used regression metrics: MAE, RMSE, and MAPE, and compares it with several representative and state-of-the-art video-based SpO2 estimation methods on the PURE and FBO datasets. Due to the limited reproducibility of existing methods, the ICU dataset is used only to evaluate the model proposed in this embodiment.

[0072] As shown in Table 1, VBOSNet achieves an MAE of 0.65 and an RMSE of 0.77 on the PURE dataset. Compared to CCSpO2Net, the MAE is reduced by 51.5% and the RMSE by 56.3%. This significant improvement fully demonstrates the robustness and accuracy of VBOSNet in SpO2 estimation in static, controlled environments.

[0073] Table 1 Experimental results on the PURE dataset

[0074] The experimental results on the FBO dataset are shown in Table 2.

[0075] Table 2 Experimental results on the FBO dataset

[0076] VBOSNet achieves a MAE of 2.74 and an RMSE of 3.53, showing a significant improvement over CNN-based methods. However, it still lags behind CCSpO2Net and LSTM-based methods. This performance difference stems from methodological differences in algorithm design and signal processing. Unlike the general-purpose approach of this embodiment, Lampier's method is specifically optimized for contact-based fingertip oxygenation measurement, thus possessing inherent advantages in this particular scenario. However, its adaptability is limited in non-contact applications or measurements at other anatomical sites. CCSpO2Net maintains optimal signal integrity through image upsampling, while the strategy of this embodiment employs spatiotemporal pooling for computational efficiency, inevitably sacrificing some controllable hemodynamic information.

[0077] On the self-built ICU dataset, VBOSNet demonstrates excellent performance, achieving an MAE of 0.61 and an RMSE of 0.95. These results highlight VBOSNet's robust feature extraction and prediction capabilities for SpO2 estimation in various scenarios. Figure 6 The paper presents a comparison and consistency analysis of the actual and predicted values ​​on three datasets, along with the corresponding Bland-Altman plots. The results show a high correlation between the predicted and actual values, further illustrating that VBOSNet exhibits consistent and robust performance across diverse scenarios.

[0078] In this embodiment, a series of ablation experiments were conducted to systematically evaluate the effectiveness of each proposed module. The results are shown in Table 3.

[0079] Table 3 Ablation Experiment Results

[0080] Introducing the MWFE module alone can bring significant performance improvements, reducing MAE and RMSE by 12.6% and 8.2% respectively on the PURE dataset; and reducing MAE and RMSE by 20.8% and 16.2% respectively on the FBO dataset, verifying the module's generalization ability to capture SpO2 spectral characterization under different measurement modes.

[0081] The OCS module exhibits more significant advantages. On the PURE dataset, it reduces MAE and RMSE by 24.1% and 14.3%, respectively; on the FBO dataset, it reduces them by 24.1% and 17.0%, respectively. These improvements demonstrate the module's significant effectiveness in learning robust, individual-independent features.

[0082] When both modules are applied simultaneously, the model exhibits optimal overall performance. On the PURE dataset, MAE and RMSE are reduced by 25.3% and 21.4%, respectively; on the FBO dataset, they are reduced by 25.9% and 21.9%, respectively. This indicates that MWFE and OCS provide complementary advantages: MWFE enhances spectral sensitivity, while OCS improves the ability to learn discriminative features. The combination of the two makes SpO2 estimation more accurate and robust under diverse conditions. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0086] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A blood oxygen saturation image comparison learning method based on multi-scene videos, characterized in that, Includes the following steps: S1. Preprocess the input video to extract physiological signal-related regions and generate a spatiotemporal map; S2. Constructing multi-wavelength feature representation based on the light absorption characteristics of hemoglobin; S3. Construct training sample pairs using a contrastive learning strategy, including anchor samples, positive samples, and negative samples; S4. Train the neural network model by combining the multi-wavelength features and contrastive learning; S5. Estimate blood oxygen saturation based on the trained model.

2. The blood oxygen saturation image comparison learning method based on multi-scene video according to claim 1, characterized in that, Step S1 includes: S11. Use detection algorithms to extract facial or fingertip areas; S12. Perform spatial downsampling and temporal window sliding to generate a spatiotemporal map.

3. The blood oxygen saturation image comparison learning method based on multi-scene video according to claim 1, characterized in that, Step S2 includes: S21. Select a target wavelength relevant to the absorption characteristics of hemoglobin; S22. Calculate the Euclidean distance between the pixel RGB value and the wavelength reference value. The calculation formula is as follows: In the formula, This represents the calculated pixel value. , and These are the reference RGB values ​​for the target wavelength, and , and These are the actual RGB values ​​of the pixels in the image; S23. Generate a multi-channel feature map based on the calculated Euclidean distance.

4. The blood oxygen saturation image comparison learning method based on multi-scene video according to claim 1, characterized in that, Step S3 includes: S31. Construct positive and negative sample pairs based on individual identity and blood oxygen saturation level, where positive samples come from different individuals but have the same blood oxygen saturation, and negative samples come from the same individual but have different blood oxygen saturation. S32. Perform data augmentation on anchor point samples to generate augmented positive samples. Augmentation operations include spatial augmentation, temporal augmentation, and color augmentation.

5. The blood oxygen saturation image comparison learning method based on multi-scene video according to claim 1, characterized in that, Step S4 includes: S41. Construct a neural network model, the model including a physiological feature extraction module, a multi-wavelength feature extraction module, and a feature fusion module; S42. Configure training parameters, including using the Adam optimizer and setting the learning rate. The weight decays to The batch size is 16, and an early stopping strategy is adopted. Training is performed for a maximum of 100 epochs. If the validation set shows no improvement for 30 consecutive epochs, the training is stopped early. S43. Define a composite loss function for model optimization, wherein the loss function includes mean squared error loss. Negative Pearson correlation coefficient loss Extended triplet loss Loss compared with multiple positive samples ; S44. Perform model training, using the composite loss function for backpropagation and parameter updates, where the overall loss is calculated by weighted summation: in, To balance the hyperparameters, which are used to adjust the contribution of each component in the joint optimization.

6. The blood oxygen saturation image comparison learning method based on multi-scene video according to claim 5, characterized in that, In S41, The physiological feature extraction module uses an improved ResNet structure combined with a Squeeze-and-Excitation attention mechanism, and its expression is as follows: In the formula, Conv represents the convolution operation, and SE represents the attention mechanism; The multi-wavelength feature extraction module employs a multi-scale channel attention mechanism, the expression of which is: In the formula, LAtt represents local channel attention, GAtt represents global channel attention, and σ is the sigmoid function. This indicates element-wise multiplication at the channel level; The feature fusion module stitches together physiological features and multi-wavelength features in the channel dimension.

7. The blood oxygen saturation image comparison learning method based on multi-scene video according to claim 5, characterized in that, Extended triplet loss in S43 Represented as: In the formula, Indicates anchor point sample, To enhance positive samples, These are positive and negative samples selected from the dataset using OCS, respectively. Let be the distance function. To control the boundary hyperparameters for the separation between positive and negative samples; Multi-positive sample contrast loss Represented as: In the formula, Indicates anchor point sample, For its two corresponding positive samples, For negative samples, Represents the computation of vectors cosine similarity, Temperature parameters for controlling the steepness of the distribution, Indicates batch size.

8. The blood oxygen saturation image comparison learning method based on multi-scene video according to claim 1, characterized in that, In step S5, a Transformer-based regressor is used to predict blood oxygen saturation values, and its expression is as follows: In the formula, FC is the fully connected layer, DeiT is the Transformer architecture, and Neck is the feature fusion module.

Images