Methods, devices, equipment, and media for monitoring high physiological stress based on rPPG

Abstract

This invention discloses a method, device, equipment, and medium for monitoring high physiological stress based on rPPG. First, a video sequence containing the user's face is acquired, and the region of interest (ROI) sequence is extracted and input into a deep learning network model. A spatiotemporal encoder extracts preliminary spatiotemporal physiological feature sequences from these sequences. Then, relying on an adaptive frequency-aware decoder, multi-scale parallel processing and dynamic feature fusion are performed on these preliminary spatiotemporal physiological feature sequences to output feature representations of abnormal high-frequency rhythmic events. Finally, a state determiner, combined with the abnormal high-frequency rhythmic event identification results, directly outputs a binary classification result indicating whether the user is in a state of high physiological stress. This invention effectively solves the problem of abnormal high-frequency rhythmic signal loss caused by existing fixed filters, achieving high-precision identification of abnormal high-frequency rhythmic events and accurately determining the user's state of high physiological stress.

Description

Technical Field

[0001] This invention relates to a method, device, equipment, and medium for monitoring high physiological stress based on rPPG, belonging to the field of physiological signal processing and deep learning technology. Background Technology

[0002] The physiological essence of overwork is the dysfunction of the autonomic nervous system (ANS), manifested as excessive excitation of the sympathetic nervous system and decreased parasympathetic nervous system activity. Long-term overwork can induce chronic diseases and even acute cardiovascular and cerebrovascular events. Traditional overwork monitoring relies heavily on subjective tools such as fatigue self-rating scales, resulting in outcomes highly influenced by personal feelings and lacking objective quantitative indicators. Improvements based on heart rate variability (HRV) are also problematic because HRV is easily affected by factors such as exercise and emotions. In high-physiological-load scenarios (e.g., high-intensity exercise, severe emotional fluctuations), HRV lacks specificity (below 70%) and has a high false alarm rate, making it difficult to provide reliable acute risk warnings. More importantly, from a physiological perspective, the sudden increase in heart rate and rhythm disturbances during abnormally high-frequency rhythmic events are themselves acute manifestations of ANS regulatory dysfunction, serving as a powerful objective marker of high physiological load. Based on this correlation, simultaneously monitoring both using the same rPPG signal source has extremely high synergistic value. However, existing technologies often treat the two as independent tasks or simple functional superpositions, without establishing a deep physiological connection and technological integration. This results in redundant system functions, fragmented judgment logic, and insufficient accuracy, failing to fully leverage the synergistic effect and making it difficult to meet the public's integrated health risk monitoring needs. Summary of the Invention

[0003] The purpose of this invention is to provide a method, device, equipment, and medium for monitoring high physiological stress based on rPPG. This invention effectively solves the problem of loss of abnormal high-frequency rhythm signals caused by existing fixed filters, achieving high-precision identification of abnormal high-frequency rhythm events, and based on this, accurately determining the user's high physiological stress state.

[0004] The technical solution of this invention: A method for monitoring high physiological stress based on rPPG, comprising the following steps: Step a: Acquire a video sequence containing the user's face using a camera, and extract the image sequence of the region of interest on the face after preprocessing; Step b: Construct a deep learning network model that includes a spatiotemporal encoder, an adaptive frequency-aware decoder, and a state determiner, and input the facial region of interest image sequence into the model; Step c: Extract preliminary spatiotemporal physiological feature sequences using a spatiotemporal encoder; Step d: The adaptive frequency-aware decoder is used to perform multi-scale parallel processing and dynamic feature fusion on the preliminary spatiotemporal physiological feature sequence to generate a fused feature representation sensitive to abnormal high-frequency rhythmic events; wherein, the multi-scale parallel processing includes processing the preliminary spatiotemporal physiological feature sequence through at least two branches with different time resolutions respectively, and the dynamic feature fusion includes adaptively determining the fusion weights of the outputs of each branch based on the dynamic characteristics of the preliminary spatiotemporal physiological feature sequence. Step e: Analyze the fused feature representation using the state determiner, and based on the identification results of abnormal high-frequency rhythmic events, directly output a binary classification result to determine whether the user is in a state of high physiological load. The deep learning network model is optimized by minimizing the total loss function, which includes main classification loss and time-frequency fidelity auxiliary loss, to improve the accuracy of monitoring high physiological load conditions.

[0005] The above-mentioned rPPG-based method for monitoring high physiological stress includes an adaptive frequency sensing decoder comprising three branches with different time resolutions: a high-resolution branch, a medium-resolution branch, and a low-resolution branch. The high-resolution branch is used to directly receive and process the preliminary spatiotemporal physiological feature sequence. The medium-resolution branch performs an average pooling operation with a step size of 2 on the preliminary spatiotemporal physiological feature sequence in the time dimension before processing. The low-resolution branch performs an average pooling operation with a step size of 4 on the preliminary spatiotemporal physiological feature sequence in the time dimension before processing.

[0006] In the aforementioned rPPG-based method for monitoring high physiological load, the adaptive frequency-sensing decoder dynamically generates fusion weights corresponding to each branch through a scale-gated attention mechanism when performing dynamic feature fusion, and the sum of each fusion weight is 1; the scale-gated attention mechanism takes the input feature sequence of the high-resolution branch as input and is processed by a fully connected layer.

[0007] In the aforementioned rPPG-based method for monitoring high physiological stress, the auxiliary loss is the mode decomposition fidelity loss, which is calculated through the following steps: Obtain the rPPG waveform predicted by the intermediate layer of the deep learning network model and the corresponding real reference waveform; Variational mode decomposition is performed on the predicted rPPG waveform and the real reference waveform respectively to obtain a set of eigenmode functions and corresponding center frequencies for each. Calculate the waveform difference loss between the corresponding intrinsic mode functions of the predicted rPPG waveform and the real reference waveform, as well as the frequency difference loss between the center frequencies; The mode decomposition fidelity loss is calculated based on the waveform difference loss and the frequency difference loss.

[0008] In the aforementioned rPPG-based method for monitoring high physiological stress, the label for high physiological stress is uniquely bound to a video data segment containing abnormal high-frequency rhythmic features in the training labels used by the state determiner during the training phase.

[0009] A device for monitoring high physiological stress based on rPPG, comprising: The video capture module is used to capture video sequences containing the user's face; The data preprocessing module is used to process the video sequence and extract facial region of interest image sequences; The model inference module, which is loaded with the aforementioned deep learning network model, is used to receive the facial region of interest image sequence and perform high physiological load state determination. The post-processing module is used to output the determination result of the model inference module.

[0010] The aforementioned apparatus, wherein the model inference module further comprises: A spatiotemporal coding unit is used to implement the function of the spatiotemporal encoder; An adaptive frequency sensing unit is used to implement the function of the adaptive frequency sensing decoder; The state determination unit is used to implement the function of the state determiner.

[0011] In the aforementioned device, the post-processing module is configured to trigger visual and / or auditory alarms when it is determined that the user is in a state of high physiological load.

[0012] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method as described above.

[0013] An electronic device includes a processor and a memory, the memory storing a computer program executable by the processor, the computer program implementing the method as described above when executed by the processor.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention designs an adaptive frequency sensing module, which can flexibly capture high-frequency signals exceeding 3 Hz during the onset of abnormal high-frequency rhythmic events by using a multi-scale parallel state space model branch and a dynamic scale-gated attention mechanism; at the same time, it combines the mode decomposition fidelity loss function to accurately constrain the time-frequency characteristics of non-stationary remote photoplethysmography (rPPG) signals, effectively preserving key physiological information.

[0015] 2. This invention uses abnormal high-frequency rhythms, an objective and quantifiable physiological event, as the basis for determining high physiological stress, replacing traditional scales that rely on subjective feelings or easily interfered heart rate variability indicators. This improves the specificity of determining high physiological stress, reduces the false alarm rate, and effectively solves the problem of unreliable judgment caused by large fluctuations and strong subjectivity of traditional monitoring methods. It provides accurate and reliable judgment basis for physiological risk warning.

[0016] 3. This invention only requires the capture of facial video through the cameras of common devices such as smartphones, computers, and smart home devices. Users do not need to wear electrodes, wristbands, or any other hardware devices, making the entire process contactless and imperceptible, effectively lowering the barrier to entry and significantly improving user compliance. Furthermore, this invention supports real-time monitoring in various scenarios, including at home, in the office, and while driving, without spatial or environmental limitations. It can flexibly match the health management needs of daily life and work, breaking through the limitations of traditional monitoring technologies and possessing broad scenario coverage capabilities. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 This is a schematic diagram of the deep learning model of the present invention; Figure 3 This diagram illustrates the core logic and signal correlation process for presenting VMD decomposition. Figure 4 This illustrates the flowchart of rPPG simulated waveform generation. Figure 5 A schematic diagram of VMD decomposition is shown; Figure 6 This diagram illustrates how VMD decomposition is used to obtain spectral information. Figure 7 This is a schematic diagram of the device of the present invention. Detailed Implementation

[0018] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this should not be construed as limiting the present invention.

[0019] Example 1: A method for monitoring high physiological stress based on rPPG, such as Figure 1 As shown, it includes the following steps: Step a: Acquire a video sequence containing the user's face (face area greater than 20%, including frame number T, vertical pixel count H, horizontal pixel count w, and color channel count C) using a camera. Preprocess the video sequence to extract facial regions of interest (ROIs) for physiological signal analysis. This step is performed by activating the device's camera, continuously capturing video frames and constructing them into a video sequence of a predetermined duration (e.g., 10 seconds). The preprocessing involves face detection and ROI cropping and normalization for each frame of the video sequence.

[0020] Step b: Construct a deep learning network model, which includes a spatiotemporal encoder, an adaptive frequency-aware decoder, and a state determiner connected in sequence; input the preprocessed facial region of interest sequence into the deep learning network model; Step c: Extract preliminary spatiotemporal physiological feature sequences from the facial region of interest image sequence using the spatiotemporal encoder; Step d: The adaptive frequency-aware decoder is used to perform multi-scale parallel processing and dynamic feature fusion on the preliminary spatiotemporal physiological feature sequence to generate a fusion feature representation sensitive to abnormal high-frequency rhythmic events; wherein, the multi-scale parallel processing includes three branches with different time resolutions to process the preliminary spatiotemporal physiological feature sequence respectively, and the dynamic feature fusion includes adaptively determining the fusion weights of the output of each branch based on the dynamic characteristics of the preliminary spatiotemporal physiological feature sequence. Step e: Analyze the fused feature representation using the state determiner, and based on the identification results of abnormal high-frequency rhythmic events, directly output a binary classification result to determine whether the user is in a state of high physiological load.

[0021] The following describes the deep learning network model in this embodiment in detail, such as... Figure 2 As shown: The spatio-temporal encoder is used to extract low-dimensional, information-dense physiological feature representations from high-dimensional, noisy video pixels.

[0022] The spatiotemporal encoder is implemented as follows: Lightweight CNN preprocessing: MobileNet (a lightweight convolutional neural network optimized for mobile devices and embedded vision applications. This model significantly reduces computation and parameter count through depthwise separable convolutions, supporting various vision tasks such as classification, detection, and segmentation) is used to extract spatial features from each frame, compressing the image patch into a feature vector, effectively reducing computation.

[0023] Mamba-Encoder (a hybrid vision model combining the Mamba state-space model and the lightweight MobileNet architecture, aiming to balance high-resolution modeling capabilities and computational efficiency): The aforementioned feature vector sequence is input into the Mamba module (a Large Language Model (LLM) architecture that primarily integrates a structured state-space sequence model to manage lengthy data sequences). Leveraging Mamba's powerful long-range dependency modeling capabilities, it learns the dynamic changes of pixel features over time, intelligently filtering out noise introduced by minute head movements and ambient light changes, and initially extracting the core information related to blood volume pulses.

[0024] The Adaptive Frequency-Aware Decoder (AFA-Mamba) is the core technology engine for solving the problem of missing anomalous high-frequency rhythmic events. It refines and amplifies dynamic features related to heart rhythm and adaptively focuses the model's "attention" on the most critical frequency ranges based on signal content, ensuring the highest capture capability for anomalous high-frequency rhythmic events. It replaces the fixed, lossy filters used in traditional signal processing with a dynamic, learnable structure. Its core is the Multi-Scale Temporal Mamba Module.

[0025] The adaptive frequency-aware decoder is implemented as follows: 1. Parallel Mamba Branching: The feature sequence output by the encoder is fed in parallel into three branches that process signals at different temporal resolutions. This multi-scale design aims to help the model better distinguish physiological signals from different sources, thereby more accurately targeting the target. Fine-Grained Branch: This branch directly processes the initial spatiotemporal physiological characteristic sequence. Maintaining the highest temporal resolution, it focuses on capturing the fine morphology of each beat of the heart and the rapid (>3Hz), irregular RR interval changes caused by abnormally high-frequency rhythmic events, making it the core of identifying abnormally high-frequency rhythmic events.

[0026] Mid-Range Branch: Before inputting into the Mamba module, the feature sequences undergo a step-2 average pooling operation in the time dimension. This branch is responsible for learning stable rhythm patterns within the normal heart rate range (1-2.5Hz), providing the model with a baseline reference for judging whether the heart rate is "normal".

[0027] Coarse-Grained Branch: This branch performs average pooling with a step size of 4 on the feature sequence. It focuses on longer-cycle physiological fluctuations such as respiration. Its role is to help the model separate heartbeat signals from other low-frequency noise such as respiration, avoiding interference and further improving the signal-to-noise ratio for identifying abnormal high-frequency rhythmic events.

[0028] 2. Scale-Gating Attention Mechanism: This is the key to achieving adaptation.

[0029] Design a small neural network consisting of several fully connected layers as a gating unit, which takes the original high-resolution feature sequence as input.

[0030] The task of this gating unit is to dynamically learn a weight vector. And the sum of the three is 1.

[0031] Working mechanism: When a sharp, rapid, non-periodic fluctuation is detected in the input sequence (a strong indication of abnormally high-frequency rhythmic events), the gating unit learns to assign weights to the high-resolution branch. Assign a very high value (e.g., 0.8). Conversely, when the signal is stable and regular, increase the weight of the medium-resolution branch. This is to confirm that this is a normal physiological state.

[0032] Feature fusion: The outputs of the medium and low resolution branches are upsampled to restore the original time length and then combined with the output of the high resolution branch. The sum is then weighted according to the dynamic weights generated by the gating unit to obtain the final frequency adaptive fusion feature that is highly sensitive to abnormal high-frequency rhythmic events.

[0033] The state determiner (also known as the AFib-EventRecognition & State Judgment Head) is the decision core of the deep learning network model. It is used to analyze the physiological characteristics of the input and directly determine whether the user is in a state of high physiological load represented by abnormal high-frequency rhythmic events.

[0034] The state determiner is implemented as follows: 1. Single decision path: This module consists of a dedicated Mamba-AFib module and a final binary classifier (such as Softmax).

[0035] 2. End-to-end training and decision-making: The entire network is trained to solve a binary classification problem. The training labels are constructed as follows: Tag 0 - "Normal Physiological State": All rPPG video clips with normal heart rhythms that do not contain abnormal high-frequency rhythmic events are tagged as this category.

[0036] Tag 1 - "High physiological stress state": All rPPG video clips with abnormal high-frequency rhythmic events are tagged as this category.

[0037] 3. Direct Conclusion Output: During inference, this module receives features from AFA-Mamba and directly outputs the probability that the user belongs to "Label 0" or "Label 1". Once determined to be "Label 1", the highest level "High Physiological Overload State / Physiological Stress" alarm is immediately triggered. In this way, the ability to identify abnormal high-frequency rhythmic events is implicitly and completely encapsulated in the task of determining acute overwork state.

[0038] Furthermore, to ensure that the deep learning network model (especially the AFA-Mamba module) can faithfully learn and reconstruct high-fidelity physiological waveforms, this embodiment abandons the traditional Fourier transform and introduces an auxiliary loss function based on variational mode decomposition (VMD). This is because abnormal high-frequency rhythmic events are typical non-stationary signals, and global analysis tools such as FFT can obscure their transient characteristics. VMD can adaptively decompose the signal into a series of modes (IMFs) with instantaneous frequencies, perfectly capturing the signal's local time-frequency structure. The specific implementation is as follows: 1. Waveform Prediction: In each training batch, the deep learning network model predicts the waveform based on the input video clips (originating from synchronously acquired ECG or high-precision PPG, named...). At the output of its physiological feature decoder (AFA-Mamba), a segment of rPPG waveform is predicted through a linear layer. .

[0039] 2. Parallel VMD decomposition: For the predicted waveform The variational mode decomposition (VMD) algorithm is applied to decompose it into K pre-defined intrinsic mode functions (IMFs): .

[0040] For real waveforms Applying the exact same VMD algorithm (using the same hyperparameters, such as the number of modes K=5, penalty factor α=2000, etc.), decompose it into the corresponding K true modes: .

[0041] 3. Multi-dimensional modal matching and loss calculation: The modal decomposition fidelity loss (MDF Loss) proposed in this embodiment is designed to consist of two parts to ensure that the predicted modes are accurately aligned with the true modes in both morphology and core frequencies: 3.1. Waveform Similarity Loss (Lwaveform): This loss penalizes the difference between the predicted mode and the true mode in the time-domain waveform. It is calculated using the root mean square error (MSE). ; 3.2. Center frequency similarity loss ( The VMD algorithm estimates a center frequency for each decomposed IMF. This loss is used to penalize the deviation between the predicted mode and the true mode at the core oscillation frequency: ; 4. Integrate into the total loss function: Ultimately, MDF Loss, as a key auxiliary regularization term, works in conjunction with the classification loss of the main task to guide network training. The total loss function is defined as: ; In the formula: It is the binary cross-entropy loss function of the main task. , and It is a hyperparameter used to balance the importance of each loss term.

[0042] In this way, the network is forced to generate an intermediate waveform that is highly consistent with the real signal in terms of its internal modal structure while striving to complete the main task of "state determination". This ensures the complete preservation and deep understanding of all key physiological information, including high-frequency abnormalities and high-frequency rhythm events.

[0043] To more clearly illustrate the core logic and signal relationships of VMD decomposition, the following supplementary explanation is provided in conjunction with the signal processing flow: Figure 3 As shown: Original signal source: The input signal of VMD decomposition consists of two parts: one is the original rPPG signal extracted after video sequence processing and ROI region detection (including ECG high-frequency information, ECG low-frequency information (main frequency), respiratory information and interference noise, low-frequency residual), and the other is the corresponding standard waveform.

[0044] Decomposition and filtering process: After processing by the AFA-Mamba module and spatiotemporal encoder, rPPG simulated waveforms are generated (e.g., Figure 4 As shown), VMD decomposition is performed on the original rPPG signal and the standard waveform respectively (e.g. Figure 5 As shown), obtain their respective spectrum information (such as...). Figure 6 As shown in the figure, the key intrinsic mode functions IMF2 and IMF3 are then selected for subsequent matching; Similarity calculation basis: Subsequent loss calculation will be based on the fusion logic of 'α×time domain similarity + (1-α)×frequency domain similarity', where α is a weight coefficient used to balance the importance of time domain and frequency domain features.

[0045] This embodiment proposes and implements an end-to-end technical solution for the abnormal high-frequency rhythm event-driven state judgment architecture (AFib). Its core is to directly determine whether a user is in a state of high physiological load by accurately identifying abnormal high-frequency rhythm events in the rPPG signal. This invention establishes a direct technical pathway from specific arrhythmia patterns to advanced physiological state determination. The adaptive frequency sensing module (AFA-Mamba) in this embodiment utilizes multi-scale Mamba and gated attention mechanisms to solve core industry pain points such as limited frequency range of existing fixed filters, energy leakage, and poor performance on non-stationary signals, achieving high-precision recognition capabilities. This embodiment introduces variational mode decomposition (VMD) technology into the loss function of deep learning, constraining the model to learn the intrinsic time-frequency structure of non-stationary physiological signals, thereby improving the fidelity of signal representation in the intermediate layers of the network.

[0046] The method of this invention improves the accuracy of acute overwork status assessment. Since there is a clear correlation between abnormal high-frequency rhythmic events and severe physiological load, this invention focuses on this strong characteristic for assessment. Compared to methods relying on conventional volatility indicators (such as HRV), it effectively improves the specificity of assessment in high physiological load scenarios, reduces false alarm rates, and enables rapid response to acute risks. Furthermore, users do not need to wear any devices; monitoring can be completed using a regular camera, resulting in high user compliance and applicability to various non-intrusive monitoring scenarios such as at home, in the office, and while driving.

[0047] Example 2: An apparatus for implementing the method described in Example 1, such as... Figure 7 As shown, the device may include: a video acquisition module 101, a data preprocessing module 102, a model inference module 103, and a post-processing module 104.

[0048] The video capture module 101 is used to invoke the device's camera hardware to capture real-time video sequences containing the user's face. In a specific implementation, this module can be a software function that calls the operating system's camera API.

[0049] The data preprocessing module 102, connected to the video acquisition module 101, is used to receive the raw video sequence. This module is responsible for detecting faces in each frame of the video sequence and cropping regions of interest (ROIs) according to preset rules (such as facial landmarks), such as the user's cheeks or forehead. The cropped ROI images are normalized (e.g., scaled to 64x64 pixels, and pixel values ​​normalized to a range) to form the standard input for subsequent modules.

[0050] The model inference module 103, connected to the data preprocessing module 102, is the core of this device. This module internally stores an end-to-end deep learning network model (PS-Mamba) for receiving and analyzing the preprocessed ROI sequences. In this embodiment, the model inference module 103 can be further divided into: The spatiotemporal coding unit 103a is responsible for extracting spatiotemporal features from the input ROI sequence. It first extracts the spatial features of each frame through a lightweight convolutional network (such as MobileNetV3), and then inputs these spatial feature vector sequences into a first-level state space model (Mamba-Encoder) to model their temporal dependencies, outputting a feature sequence containing preliminary physiological information.

[0051] The adaptive frequency sensing unit 103b is responsible for refining the feature sequence output by the spatiotemporal coding unit 103a. This unit includes multiple parallel second-level state-space models (i.e., high, medium, and low resolution branches) that process different temporal resolutions, as well as a scale-gated attention mechanism. This mechanism dynamically generates weights based on the dynamic characteristics of the input features, and performs weighted fusion of the outputs from multiple branches to obtain fused features sensitive to high-frequency signals.

[0052] State determination unit 103c: Responsible for making the final binary classification judgment based on the fused features output by adaptive frequency sensing unit 103b. This unit consists of a third-level state space model (Mamba-AFib) and a Softmax classifier, directly outputting the probability that the user is in a "normal physiological state" or a "high physiological load state".

[0053] The post-processing module 104 is connected to the model inference module 103. This module receives the probability value output by the state determination unit 103c. When the probability of "high physiological load state" exceeds a preset threshold (e.g., 0.8), the module displays a high-risk alarm on the user interface of the user device or sounds an audible alarm via a buzzer.

[0054] Example 3: This example provides an electronic device that implements the method of Example 1. The electronic device is in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit, at least one storage unit, a bus connecting different system components (including the storage unit and the processing unit), and a display unit.

[0055] The storage unit stores program code, which can be executed by the processing unit to perform the steps described in the method section of Embodiment 1 above, according to various exemplary embodiments of the present invention.

[0056] The storage unit may include readable media in the form of volatile storage units, such as random access memory (RAM) and / or cache storage units, and may further include read-only memory (ROM).

[0057] The storage unit may also include a program / utility having a set (at least one) of program modules, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0058] A bus can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus that uses any of the various bus structures.

[0059] The electronic device can also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output (I / O) interfaces. Furthermore, the electronic device can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. Further, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0060] From the above description of the embodiments, those skilled in the art will readily understand that the embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0061] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, having stored thereon a program product capable of implementing the system of this specification. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section above.

[0062] In summary, this invention can effectively solve the problem of loss of abnormal high-frequency rhythm signals caused by existing fixed filters, achieve high-precision identification of abnormal high-frequency rhythm events, and accurately determine the user's high physiological load state based on this.

Claims

1. A method for monitoring high physiological stress based on rPPG, characterized in that, Includes the following steps: Step a: Acquire a video sequence containing the user's face using a camera, and extract the image sequence of the region of interest on the face after preprocessing; Step b: Construct a deep learning network model that includes a spatiotemporal encoder, an adaptive frequency-aware decoder, and a state determiner, and input the facial region of interest image sequence into the model; Step c: Extract preliminary spatiotemporal physiological feature sequences using a spatiotemporal encoder; Step d: The adaptive frequency-aware decoder is used to perform multi-scale parallel processing and dynamic feature fusion on the preliminary spatiotemporal physiological feature sequence to generate a fused feature representation sensitive to abnormal high-frequency rhythmic events; wherein, the multi-scale parallel processing includes processing the preliminary spatiotemporal physiological feature sequence through at least two branches with different time resolutions respectively, and the dynamic feature fusion includes adaptively determining the fusion weights of the outputs of each branch based on the dynamic characteristics of the preliminary spatiotemporal physiological feature sequence. Step e: Analyze the fused feature representation using the state determiner, and based on the identification results of abnormal high-frequency rhythmic events, directly output a binary classification result to determine whether the user is in a state of high physiological load. The deep learning network model is optimized by minimizing the total loss function, which includes main classification loss and time-frequency fidelity auxiliary loss, to improve the accuracy of monitoring high physiological load conditions.

2. The method for monitoring high physiological stress based on rPPG according to claim 1, characterized in that, The adaptive frequency-aware decoder includes three branches with different time resolutions: a high-resolution branch, a medium-resolution branch, and a low-resolution branch. The high-resolution branch is used to directly receive and process the preliminary spatiotemporal physiological feature sequence. The medium-resolution branch performs an average pooling operation with a step size of 2 on the preliminary spatiotemporal physiological feature sequence in the time dimension before processing. The low-resolution branch performs an average pooling operation with a step size of 4 on the preliminary spatiotemporal physiological feature sequence in the time dimension before processing.

3. The method for monitoring high physiological stress based on rPPG according to claim 2, characterized in that, When the adaptive frequency-aware decoder performs dynamic feature fusion, it dynamically generates fusion weights corresponding to each branch through a scale-gated attention mechanism, and the sum of each fusion weight is 1. The scale-gated attention mechanism takes the input feature sequence of the high-resolution branch as input and is processed by a fully connected layer.

4. The method for monitoring high physiological stress based on rPPG according to claim 1, characterized in that, The auxiliary loss is the mode decomposition fidelity loss, which is calculated through the following steps: Obtain the rPPG waveform predicted by the intermediate layer of the deep learning network model and the corresponding real reference waveform; Variational mode decomposition is performed on the predicted rPPG waveform and the real reference waveform respectively to obtain a set of eigenmode functions and corresponding center frequencies for each. Calculate the waveform difference loss between the corresponding intrinsic mode functions of the predicted rPPG waveform and the real reference waveform, as well as the frequency difference loss between the center frequencies; The mode decomposition fidelity loss is calculated based on the waveform difference loss and the frequency difference loss.

5. The method for monitoring high physiological stress based on rPPG according to claim 1, characterized in that, In the training labels used by the state determiner during the training phase, the label for high physiological load state is uniquely bound to the video data segment containing abnormal high-frequency rhythm features.

6. A device for monitoring high physiological stress based on rPPG, characterized in that, include: The video capture module is used to capture video sequences containing the user's face; The data preprocessing module is used to process the video sequence and extract facial region of interest image sequences; The model inference module, which is internally loaded with a deep learning network model as described in any one of claims 1 to 5, is used to receive the facial region of interest image sequence and perform high physiological load state determination; The post-processing module is used to output the determination result of the model inference module.

7. The apparatus according to claim 6, characterized in that, The model inference module further includes: A spatiotemporal coding unit is used to implement the function of the spatiotemporal encoder; An adaptive frequency sensing unit is used to implement the function of the adaptive frequency sensing decoder; The state determination unit is used to implement the function of the state determiner.

8. The apparatus according to claim 6 or 7, characterized in that, The post-processing module is configured to trigger visual and / or auditory alarms when it is determined that the user is in a state of high physiological load.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

10. An electronic device comprising a processor and a memory, characterized in that, The memory stores a computer program executable by the processor, which, when executed by the processor, implements the method as described in any one of claims 1 to 5.

Images