Non-contact pressure and fatigue state discrimination method with dual-indicator joint reasoning

Abstract

The application provides a non-contact pressure and fatigue state identification method based on double-indicator joint reasoning, and relates to the field of psychophysiological analysis. The method comprises the following steps: collecting a facial visible light video and a first EDA signal; extracting an rPPG signal and a first facial behavior indicator sequence; processing the signal amplitudes of the first EDA signal and the first facial behavior indicator sequence to obtain a target EDA signal and a target behavior indicator sequence; inputting the rPPG signal and the target behavior indicator sequence into a fatigue detection model to obtain fatigue information; inputting the rPPG signal and the target EDA signal into a psychological pressure detection model to obtain psychological pressure information; and synchronously outputting and displaying the fatigue information and the psychological pressure information based on a preset signal waveform comparison display area. The application simultaneously analyzes the psychological pressure state and the fatigue state, comprehensively considers the double indicators, identifies cross-scene state fluctuations, and can effectively improve the accuracy and efficiency of the analysis of the psychological pressure state and the fatigue state.

Description

Technical Field

[0001] This application relates to the field of psychophysiological analysis technology, specifically to a non-contact method for identifying stress and fatigue states through dual-signal joint reasoning. Background Technology

[0002] Psychological stress state recognition is an important research area in affective computing. Fatigue refers to the condition caused by the cumulative effect of training load and possible non-training stress (such as psychological, social, nutritional, and sleep factors), which exceeds the individual's ability to adapt and recover.

[0003] Based on the principle of synchronicity in emotional responses in psychophysiology, the activation of an individual's autonomic nervous system (e.g., quantified by the EDA index of skin conductance) is highly correlated in time with the involuntary micro-movements of facial muscles (i.e., facial motor units). Therefore, physiological and behavioral indicators are not isolated from each other. Fatigue is reflected at both the physiological and behavioral levels, while psychological stress is primarily reflected at the physiological level. Fatigue and psychological stress are important indicators for individual assessment. Synchronous analysis of fatigue and psychological stress can comprehensively reflect an individual's overall condition.

[0004] In related technologies, multimodal signals are collected for analysis when analyzing fatigue and psychological stress. However, due to the long signal sequences and noise, it is difficult to directly capture key information by directly inputting the signals into the model. How to simultaneously analyze physiological and behavioral indicators and perform dual-indicator identification to improve the detection accuracy of psychological stress and fatigue is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application provides a non-contact method for identifying stress and fatigue states through dual-indicator joint reasoning, which solves the problems of insufficient accuracy and efficiency in current analysis of psychological stress and fatigue states.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] In a first aspect, embodiments of this application provide a non-contact stress and fatigue state identification method based on dual-signal joint reasoning. This method includes: acquiring visible light video of a face via a camera and acquiring a first EDA signal via a wrist-worn wearable device; extracting an rPPG signal and a first facial behavioral indicator sequence based on the visible light video of the face; the rPPG signal and the first EDA signal being physiological indicator information; resampling and sorting the signal amplitudes of the first EDA signal and the first facial behavioral indicator sequence to obtain an aligned second EDA signal and a second facial behavioral indicator sequence; performing cross-weighting processing on the second EDA signal and the second facial behavioral indicator sequence to obtain a target EDA signal and a target behavioral indicator sequence; inputting the rPPG signal and the target behavioral indicator sequence into a preset fatigue detection model to obtain fatigue information; the fatigue detection model being an integrated classification model for non-contact detection of exercise fatigue; inputting the rPPG signal and the target EDA signal into a preset psychological stress detection model to obtain psychological stress information; and synchronously outputting and displaying the fatigue information and psychological stress information based on a preset signal waveform comparison display area.

[0008] According to a first aspect of the embodiments of this application, the aforementioned cross-weighting processing of the second EDA signal and the second facial behavior indicator sequence to obtain the target EDA signal and the target behavior indicator sequence may specifically include the following steps: weighting the second facial behavior indicator sequence based on the second EDA signal using a preset weighting strategy, and obtaining the target behavior indicator sequence through cross-modal information interaction; and weighting the second EDA signal based on the second facial behavior indicator sequence using a weighting strategy, and obtaining the target EDA signal through cross-modal information interaction.

[0009] According to a first aspect of the embodiments of this application, the aforementioned weighting strategy is used to weight the second facial behavioral indicator sequence based on the second EDA signal to obtain the target behavioral indicator sequence through cross-modal information interaction. Specifically, it may include the following steps: determining a scalar indicator sequence M(t) based on the time sequence of the second EDA signal to measure the individual's physiological arousal level in real time; the scalar indicator sequence M(t) is the physiological signal energy, peak quantity, or more specifically, a measure with clear physiological significance within a time window; converting the scalar indicator sequence M(t) into a time-varying attention weight sequence w(t) through a pre-set, nonlinear deterministic attention weight mapping function f; wherein, the deterministic attention weight mapping function f is used to map the absolute value or relative change of the physiological indicator to a weighted interval normalized to [0,1]; using the attention weight sequence w(t), performing a multiplicative gain control or element-level weighting operation on the synchronously acquired second facial behavioral indicator sequence to generate the target behavioral indicator sequence.

[0010] According to a first aspect of the embodiments of this application, the calculation process of the attention weight sequence w(t) satisfies the expression:

[0011]

[0012] Where μ is a preset physiological index threshold, and σ is a scaling factor that controls the transition sharpness.

[0013] According to a first aspect of the embodiments of this application, the calculation process of the target behavior indicator sequence satisfies the expression:

[0014] AU enhanced (t)=G(w(t))·AU(t)

[0015] Among them, AU enhanced (t) represents the target behavior indicator sequence, G(w(t)) is a gain function with w(t) as input, G(w(t))=1+α·w(t), α is the preset modulation intensity coefficient, and AU(t) is the second facial behavior indicator sequence.

[0016] According to a first aspect of the embodiments of this application, the aforementioned resampling and sorting of the signal amplitudes of the first EDA signal and the first facial behavior indicator sequence to obtain an aligned second EDA signal and a second facial behavior indicator sequence may specifically include the following steps: A target synchronization frequency is preset; based on the start and end timestamps of the signal segments to be processed, a standard, uniformly distributed target time vector is generated as a unified time reference for synchronizing all signals; wherein, the first EDA signal and the first facial behavior indicator sequence are both signal segments to be processed; a continuous interpolation function is constructed using the timestamp sequence of the first EDA signal and the first facial behavior indicator sequence as independent variables and the amplitude sequence as dependent variables; the type of the interpolation function is selected according to the signal characteristics and processing accuracy requirements, and it can estimate the signal amplitude at any time based on discrete original data points; the interpolation function is used to calculate the resampled signal amplitudes corresponding to the first EDA signal and the first facial behavior indicator sequence at each time point of the target time vector; all calculated signal amplitudes are arranged in order, and the two resampled discrete time signal sequences are determined by the arrangement to obtain the second EDA signal and the second facial behavior indicator sequence.

[0017] According to a first aspect of the embodiments of this application, the training process of the aforementioned fatigue detection model includes: acquiring a first facial visible light video and a second facial visible light video corresponding to a first resting state and a second resting state, respectively, and extracting the corresponding physiological features and facial behavior features; using the physiological features and facial behavior features corresponding to the first resting state as baseline features, using the physiological features and facial behavior features corresponding to the second resting state as instantaneous state features, and using the difference between the instantaneous state features and the baseline features as baseline relative features; performing feature selection on the set of motion fatigue features including instantaneous state features and baseline relative features based on domain knowledge and statistical tests to obtain a key feature set; training multiple base learners based on the key feature set, using ensemble learning and Bayesian optimization to determine the optimal weight combination of the base learners, and obtaining the fatigue detection model.

[0018] According to a first aspect of the embodiments of this application, the first facial behavior indicator sequence includes a head posture sequence, an eye gaze angle sequence, and a facial behavior unit feature sequence; the head posture sequence describes the motion state of the head through translation and rotation in three-dimensional space; the eye gaze angle sequence describes the overall change in the eye gaze direction by capturing the displacement of the eyes in the horizontal and vertical directions, combined with horizontal deflection and vertical deflection; the actions corresponding to the facial behavior unit feature sequence include: raising the inner eyebrow, raising the outer eyebrow, lowering the eyebrow, raising the upper eyelid, raising the cheek, tightening the eyelid, wrinkling the nose, raising the upper lip, pulling up the corner of the mouth, tightening the corner of the mouth, lowering the corner of the mouth, raising the chin, stretching the lips, tightening the lips, opening the lips, opening the jaw, sucking the lips, and blinking.

[0019] Secondly, embodiments of this application provide a non-contact stress and fatigue state identification system based on dual-signal joint reasoning. This non-contact stress and fatigue state identification system based on dual-signal joint reasoning includes: an acquisition module, an extraction module, an alignment module, a weighting module, a fatigue detection module, a psychological stress detection module, and a synchronous output module.

[0020] Specifically, the acquisition module is used to acquire visible light video of the face via a camera and acquire a first EDA signal via a wrist-worn wearable device; the extraction module is used to extract the rPPG signal and a first facial behavioral indicator sequence based on the visible light video of the face; the rPPG signal and the first EDA signal are physiological indicator information; the alignment module is used to resample and sort the signal amplitudes of the first EDA signal and the first facial behavioral indicator sequence to obtain an aligned second EDA signal and a second facial behavioral indicator sequence; the weighting module is used to perform cross-weighting processing on the second EDA signal and the second facial behavioral indicator sequence to obtain a target EDA signal and a target behavioral indicator sequence; the fatigue detection module is used to input the rPPG signal and the target behavioral indicator sequence into a preset fatigue detection model to obtain fatigue information; the fatigue detection model is an integrated classification model for non-contact detection of exercise fatigue; the psychological stress detection module is used to input the rPPG signal and the target EDA signal into a preset psychological stress detection model to obtain psychological stress information; the synchronous output module is used to synchronously output and display fatigue information and psychological stress information based on a preset signal waveform comparison display area.

[0021] Thirdly, embodiments of this application provide an electronic device, which includes: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the non-contact pressure and fatigue state identification method of dual-signal joint reasoning described in the first aspect.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program or instructions that, when executed by a processor, implement the non-contact pressure and fatigue state identification method of dual-signal joint reasoning described in the first aspect.

[0023] This application provides a non-contact method for identifying pressure and fatigue states using dual-signal joint reasoning. Compared with existing technologies, it has the following advantages:

[0024] This application aims to jointly analyze psychological stress and fatigue states. It acquires facial visible light video and a first EDA signal, extracting rPPG signals and a first facial behavioral indicator sequence. To address the issues of excessively long and noisy signal sequences, the signal amplitudes of the first EDA signal and the first facial behavioral indicator sequence are resampled and sorted to achieve signal alignment. Since the rPPG signal and the first facial behavioral indicator sequence are inherently aligned, after the aforementioned resampling and sorting, the rPPG signal, the second EDA signal, and the second facial behavioral indicator sequence are aligned. Because physiological and behavioral indicators are interrelated, after obtaining multimodal data through signal extraction and alignment, this application performs cross-weighting processing on the second EDA signal and the second facial behavioral indicator sequence. This dynamic weighting of intrinsic physiological signals and facial behavioral features enables interpretable, efficient, and direct cross-modal information interaction without relying on iterative gradient optimization training using large-scale labeled data. Furthermore, fatigue and psychological stress information are obtained through model analysis and simultaneously output for comparative analysis. This application analyzes both psychological stress and fatigue states simultaneously, using dual indicators for comprehensive consideration to identify cross-scenario state fluctuations, which can effectively improve the accuracy and efficiency of psychological stress and fatigue state analysis. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating a non-contact pressure and fatigue state identification method based on dual-signature joint reasoning provided in an embodiment of this application.

[0027] Figure 2 yes Figure 1 An exemplary partial process diagram of S140;

[0028] Figure 3 This is a schematic diagram of the structure of a non-contact pressure and fatigue state identification system based on dual-signal joint reasoning provided in an embodiment of this application;

[0029] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0032] This application provides a non-contact method for identifying stress and fatigue states through dual-indicator joint reasoning, which solves the problems of insufficient accuracy and efficiency in the current analysis of psychological stress and fatigue states.

[0033] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0034] The following section first introduces a non-contact method for identifying pressure and fatigue states using dual-signal joint reasoning, provided in the embodiments of this application.

[0035] This application provides a flowchart illustrating a non-contact pressure and fatigue state identification method based on dual-signal joint reasoning, as shown in the embodiments below. Figure 1 As shown, the non-contact pressure and fatigue state identification method based on dual-indicator joint reasoning may include the following steps S110-S170.

[0036] S110. Acquire visible light video of the face through a camera and acquire the first EDA signal through a wrist-worn wearable device.

[0037] S120. Based on the visible light video of the face, extract the rPPG signal and the first facial behavioral indicator sequence; the rPPG signal and the first EDA signal are physiological indicator information.

[0038] S130. The signal amplitudes of the first EDA signal and the first facial behavior indicator sequence are resampled and sorted to obtain an aligned second EDA signal and a second facial behavior indicator sequence.

[0039] S140. Perform cross-weighting on the second EDA signal and the second facial behavior indicator sequence to obtain the target EDA signal and the target behavior indicator sequence.

[0040] S150. Input the rPPG signal and target behavior indicator sequence into the preset fatigue detection model to obtain fatigue information; the fatigue detection model is an integrated classification model for non-contact detection of motion fatigue.

[0041] S160. Input the rPPG signal and the target EDA signal into the preset psychological stress detection model to obtain psychological stress information.

[0042] S170: Based on the preset signal waveform comparison display area, fatigue information and psychological stress information are output and displayed synchronously.

[0043] The above is a specific implementation of the non-contact stress and fatigue state identification method provided in this application for dual-signal joint reasoning. It can be understood that this application performs joint analysis of psychological stress state and fatigue state, collects facial visible light video and first EDA signal and extracts rPPG signal and first facial behavior indicator sequence; in order to solve the problem of excessively long signal sequence and noise, the signal amplitude of the first EDA signal and the first facial behavior indicator sequence are resampled and sorted to complete signal alignment; since the rPPG signal and the first facial behavior indicator sequence are already aligned, after the aforementioned resampling and sorting are completed, the rPPG signal, the second EDA signal and the second facial behavior indicator sequence are aligned.

[0044] Furthermore, since physiological and behavioral indicators are interrelated, this application, after extracting and aligning signals to obtain multimodal data, performs cross-weighting processing on the second EDA signal and the second facial behavioral indicator sequence. This involves dynamically weighting the intrinsic physiological signals and facial behavioral features to achieve interpretable, efficient, and direct cross-modal information interaction without relying on large-scale labeled data for iterative gradient optimization training. Subsequently, fatigue and psychological stress information are obtained through model analysis and simultaneously output for comparative analysis. This application analyzes both psychological stress and fatigue states simultaneously, utilizing dual indicators for comprehensive consideration, effectively improving the accuracy and efficiency of psychological stress and fatigue state analysis.

[0045] In one example, the first facial behavior indicator sequence includes a head posture sequence, an eye gaze angle sequence, and a facial behavior unit feature sequence. The head posture sequence describes the head's motion state through translation and rotation in three-dimensional space. The eye gaze angle sequence describes the overall change in eye gaze direction by capturing the displacement of the eyes in the horizontal and vertical directions, combined with horizontal and vertical deflection. The facial behavior unit feature sequence corresponds to the following actions: raising the inner eyebrow, raising the outer eyebrow, lowering the eyebrow, raising the upper eyelid, raising the cheek, tightening the eyelid, wrinkling the nose, raising the upper lip, pulling up the corners of the mouth, tightening the corners of the mouth, lowering the corners of the mouth, raising the chin, stretching the lips, tightening the lips, opening the lips, opening the jaw, sucking the lips, and blinking.

[0046] In some embodiments, the aforementioned resampling and sorting of the signal amplitudes of the first EDA signal and the first facial behavior indicator sequence to obtain an aligned second EDA signal and a second facial behavior indicator sequence, i.e., the aforementioned S130 may specifically include the following steps:

[0047] S210. A target synchronization frequency is preset, and a standard, uniformly distributed target time vector is generated based on the start and end timestamps of the signal segment to be processed, so as to serve as a unified time reference for the synchronization of all signals; wherein, the first EDA signal and the first facial behavior indicator sequence are both signal segments to be processed.

[0048] S220. Using the timestamp sequence of the first EDA signal and the first facial behavior indicator sequence as independent variables and the amplitude sequence as dependent variables, a continuous interpolation function is constructed. The type of interpolation function is selected according to the signal characteristics and processing accuracy requirements, and it can estimate the signal amplitude at any time based on discrete original data points.

[0049] S230. The interpolation function is used to evaluate the target time vector at each time point, and the resampled signal amplitude corresponding to the first EDA signal and the first facial behavior indicator sequence is calculated.

[0050] S240. Arrange all the calculated signal amplitudes in order, and determine the two discrete-time signal sequences after resampling by arranging them to obtain the second EDA signal and the second facial behavior indicator sequence.

[0051] In the embodiments of this application, it is understood that the application performs frame rate alignment, distance alignment, and location alignment on the data to generate a discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension. Adjacent elements in the target time vector have fixed time intervals, and the target time vector constitutes a unified time reference upon which all signals rely for synchronization. The resampled second EDA signal and the second facial behavior indicator sequence ensure the synchronicity and consistency of the system output signal in the time dimension, guaranteeing the synchronicity and accuracy of subsequent model analysis.

[0052] In some embodiments, the aforementioned cross-weighting of the second EDA signal and the second facial behavior indicator sequence yields the target EDA signal and the target behavior indicator sequence. Specifically, S140 may include the following steps:

[0053] S310. Using a preset weighting strategy, the second facial behavior indicator sequence is weighted based on the second EDA signal, and the target behavior indicator sequence is obtained through cross-modal information interaction.

[0054] S320. Based on the second facial behavior indicator sequence, the second EDA signal is weighted using a weighting strategy, and the target EDA signal is obtained through cross-modal information interaction.

[0055] In this embodiment, the second EDA signal corresponds to physiological indicators, and the second facial behavioral indicator sequence corresponds to behavioral indicators. It is understood that, based on the principle of synchronicity in emotional responses in psychophysiology, the activation of an individual's autonomic nervous system (e.g., quantified by EDA indicators of skin conductance) is highly temporally correlated with the difficult-to-inhibit micro-movements of facial muscles (i.e., facial motor units). Therefore, after extracting multiple physiological behavioral modalities, the intrinsic physiological signals and facial behavioral features are dynamically weighted together, focusing on key information, reducing noise interference, and fully utilizing the relationship between the two indicators to obtain more representative and accurate data.

[0056] In some embodiments, please refer to Figure 2 The aforementioned method uses a preset weighting strategy to weight the second facial behavior indicator sequence based on the second EDA signal, and obtains the target behavior indicator sequence through cross-modal information interaction. Specifically, the aforementioned S310 may include the following steps:

[0057] S410. Based on the time series of the second EDA signal, determine a scalar index sequence M(t) to measure the individual's physiological arousal level in real time; the scalar index sequence M(t) is the physiological signal energy, number of peaks, or a more specific measure with clear physiological significance within a time window;

[0058] S420. The scalar index sequence M(t) is transformed into a time-varying attention weight sequence w(t) through a pre-defined, non-linear deterministic attention weight mapping function f; wherein, the deterministic attention weight mapping function f is used to map the absolute value or relative change of the physiological index to a weight interval normalized to [0,1].

[0059] S430. Using the attention weight sequence w(t), perform a multiplicative gain control or element-level weighting operation on the synchronously acquired second facial behavior indicator sequence to generate the target behavior indicator sequence.

[0060] In the embodiments of this application, it is understood that the physiological feature sequence includes, but is not limited to, the electrical activity of the skin (EDA) feature sequence or the remote photoplethysmography (rPPG) feature sequence. This application first calculates a scalar index sequence M(t) from the time-series sequence of the second EDA signal, which can quantitatively reflect an individual's physiological arousal level in real time. The scalar index sequence M(t) can be defined as the physiological signal energy, peak quantity, or more specifically, a measure with clear physiological significance, such as the baseline drift of the skin conductance level (SCL), the peak density of skin conductance responses (SCRs), or the heart rate (HR) and its heart rate variability (HRV) index extracted from the rPPG signal. Secondly, the calculated scalar index sequence M(t) is converted into a time-varying attention weight sequence w(t) through a pre-defined, non-linear deterministic attention weight mapping function f. The choice of the mapping function f aims to map the absolute or relative changes of physiological indicators to a weighted interval normalized to [0,1]. The modeling adopts a "threshold effect" (i.e., the behavioral meaning is enhanced when physiological arousal exceeds a certain level) and uses a Sigmoid gating function. This step aims to capture the activation state of the autonomic nervous system as an indicator of facial behavior.

[0061] Finally, this application utilizes the generated attention weight sequence w(t) to perform a multiplicative gain control or element-wise weighting operation on the synchronously acquired behavioral feature sequences, such as facial behavioral unit feature sequences, head pose sequences, and eye gaze angle sequences, to generate the final enhanced behavioral feature sequence, such as the enhanced sequence for facial motion units. Through this mechanism, interpretable, efficient, and direct cross-modal information interaction is achieved, and the entire process does not rely on iterative gradient optimization training using large-scale labeled data.

[0062] In one example, the calculation process of the aforementioned attention weight sequence w(t) satisfies the expression:

[0063]

[0064] Where μ is a preset physiological index threshold, and σ is a scaling factor that controls the sharpness of the transition.

[0065] The calculation process of the aforementioned target behavior indicator sequence satisfies the expression:

[0066] AU enhanced (t)=G(w(t))·AU(t)

[0067] Among them, AU enhanced (t) represents the target behavior indicator sequence, G(w(t)) is a gain function with w(t) as input, G(w(t))=1+α·w(t), α is the preset modulation intensity coefficient, and AU(t) is the second facial behavior indicator sequence.

[0068] In some embodiments, the training process of the fatigue detection model includes:

[0069] S510. Acquire the first and second facial visible light videos corresponding to the first and second resting states, respectively, and extract the corresponding physiological features and facial behavioral features. It can be understood that the first resting state can be the user's pre-exercise resting state, and the second resting state can be the resting state the day after exercise. Considering the characteristics of exercise fatigue, setting the detection time points to before and the day after exercise allows for better monitoring of exercise fatigue caused by the cumulative effects of high-intensity exercise or long-term exercise, providing a technical basis for more comprehensive fatigue management.

[0070] Specifically, in this step, for example, a camera can be used to capture 90 seconds of visible light video of the user's face (resting state before exercise and resting state the day after exercise), with the time interval between the two captures set to 20 hours or more. For data collection requirements, normal lighting conditions can be set, and the user should maintain a natural facial expression, avoid significant head movement, and keep their face approximately 40-60cm away from the camera during video capture.

[0071] This application establishes a motion fatigue feature set that integrates physiological and facial behavioral characteristics. It innovatively combines physiological features (such as heart rate variability, including SDNN and RMSSD) and facial behavioral features (such as head posture, gaze angle, and facial motor unit AU) to construct a multi-index motion fatigue feature set, improving the comprehensiveness and accuracy of detection. The specific content regarding "extracting corresponding physiological and facial behavioral features" involves the following:

[0072] This invention defines the physiological features as including heart rate (HR) and heart rate variability (HRV). Specifically, based on acquired facial visible light video and locating the region of interest (ROI), a physiological signal processing algorithm is used to extract and filter the remote photoplethysmographic (rPPG) signal from the facial visible light video. A peak detection algorithm is then used to perform spectral analysis on the processed rPPG signal, and the heart rate (HR) is calculated by combining the signal's sampling frequency. Noise is eliminated while retaining the frequency range relevant to heart rate, and the filtered signal is then used to calculate the key heart rate variability (HRV) feature using the HeartPy heart rate estimation algorithm.

[0073] Facial behavioral features involve two processes: feature extraction and feature construction. Feature extraction can be performed as follows: For example, based on the acquired first and second facial visible light videos, a face recognition algorithm (e.g., OpenFace) is used to detect the face and locate 68 facial key points. Based on these key points, temporal data of facial behavior, such as head pose, eye gaze direction, and facial motion unit (AU), are obtained, specifically including:

[0074] Pose: The motion of the head is described by translation and rotation in three-dimensional space. The translation components (Tx, Ty, Tz) correspond to the displacement of the head along the X-axis (left and right), Y-axis (vertical), and Z-axis (front and back) in the Cartesian coordinate system, respectively. The rotation components (Rx, Ry, Rz) represent the pitch angle (nodding), yaw angle (shaking head), and roll angle (tilting head) of the head in Euler angles.

[0075] Gaze: By capturing the displacement of the eye in the horizontal direction (e.g., gaze_1_x) and the vertical direction (e.g., gaze_1_y), it reflects the eye's shift to the left or right or up and down; at the same time, it comprehensively describes the overall change in the eye's gaze direction through the angle information of gaze_angle, which combines the components of horizontal deflection (gaze_yaw) and vertical deflection (gaze_pitch).

[0076] AU: Inner eyebrow raised (AU01), outer eyebrow raised (AU02), eyebrow lowered (AU04), upper eyelid raised (AU05), cheek raised (AU06), eyelid tightened (AU07), nose wrinkled (AU09), upper lip raised (AU10), corner of mouth pulled up (AU12), corner of mouth tightened (AU14), corner of mouth lowered (AU15), chin raised (AU17), lips stretched (AU20), lips tightened (AU23), lips open (AU25), jaw open (AU26), lips sucking (AU28), blinking (AU45); where each AU includes the existence of AU (AU_c, binary classification, 0 or 1 indicates existence) and the strength of AU (AU_r, regression value, usually 0 to 5).

[0077] Regarding feature construction, the extracted original Pose, Gaze, and AU can be considered as sequence data extracted from each frame of a facial visible light video. However, this approach suffers from drawbacks such as the susceptibility of raw frame-level data to transient interference (e.g., AU detection errors caused by sudden changes in illumination) and the inability of discrete frame features to characterize behavioral patterns. To extract low-dimensional, robust, and interpretable facial behavioral features from the high-dimensional, redundant raw sequence data, statistical features for each facial behavioral sequence are calculated, including the mean, standard deviation, minimum, maximum, median, first quartile (25%), and third quartile (75%), which are then used as the final facial behavioral features.

[0078] S520. The physiological characteristics and facial behavioral characteristics corresponding to the first resting state are used as baseline features, and the physiological characteristics and facial behavioral characteristics corresponding to the second resting state are used as immediate state features. The difference between the immediate state features and the baseline features is used as the baseline relative features. It is understood that this embodiment of the invention considers the existence of universal physiological responses under exercise fatigue, such as changes in heart rate variability and drowsiness. Furthermore, due to significant differences in physiological conditions, training levels, genetic factors, and psychological states among individuals, the forms and severity of exercise fatigue manifestations vary among different users. Therefore, to comprehensively consider both universal physiological responses and individual differences, this embodiment of the invention establishes a set of exercise fatigue features that considers both immediate features and baseline relative features.

[0079] S530. Based on domain knowledge and statistical tests, feature selection is performed on the set of motion fatigue features, including immediate state features and baseline relative features, to obtain a key feature set. It is understood that after the processing in step S520, this embodiment of the invention obtains a set of motion fatigue features including the aforementioned immediate state features and baseline relative features. Based on this, this embodiment of the invention considers that feature selection methods relying solely on statistical tests may miss important features with clinical significance but insufficient statistical power, while feature selection relying solely on prior domain knowledge is unlikely to capture new, under-studied potential features, thus limiting the comprehensiveness of feature selection. Therefore, this embodiment of the invention integrates domain knowledge to determine key features of motion fatigue and combines statistical test-based feature selection methods to mine new features related to motion fatigue from facial features, ultimately constructing a feature set for motion fatigue detection.

[0080] Specifically: (1) Selection of motion fatigue features based on domain knowledge:

[0081] Related technologies indicate that the autonomic regulation of the heart changes during exercise fatigue. Heart rate variability typically decreases due to decreased parasympathetic activity and increased sympathetic activity. Head posture may droop or tilt, eye focus may become unfocused, and frequent blinking may occur. Therefore, by summarizing existing literature on exercise fatigue, it is clear that heart rate characteristics such as HR, SDNN, and RMSSD, as well as facial behavior characteristic AU25, are all significantly related to exercise fatigue.

[0082] (2) Selection of exercise fatigue characteristics based on statistical tests:

[0083] To fully explore the linear and nonlinear relationships between the aforementioned set of sports fatigue features and sports fatigue, a statistical test was used for feature selection, including F-test, false discovery rate correction, and mutual information screening. The relevant details are as follows:

[0084] First, all features in the exercise fatigue feature set were normalized to ensure the data were on the same scale. Next, the F-test was used to evaluate the correlation between each feature and the label, calculating the F-value and the original p-value to quantify the statistical significance of the feature. To avoid excessively high false positive rates due to multiple testing, a false discovery rate (FDR) correction was applied, retaining only features with an adjusted p-value < 0.05, thus controlling false positive errors. Simultaneously, the mutual information between the feature and the label was calculated, setting a threshold of 0.05 to filter out features with high mutual information. This allows for the retention of features highly correlated with the label while eliminating interference from noisy features.

[0085] Subsequently, an initial feature set is formed by taking the union of significant features and highly mutual information features using the F-test, ensuring that the selected features perform well in both statistical significance and information gain. Next, a correlation matrix of the initial feature set is constructed to identify highly correlated feature pairs with an absolute value > 0.9, and these are used as undirected edges to construct an undirected graph of features. The depth-first search (DFS) algorithm is used to traverse this undirected graph to determine connected components, i.e., highly correlated feature groups. In each feature group, the feature with the smallest original p-value is selected as the representative to eliminate redundancy; ungrouped individual features are directly retained.

[0086] Finally, a set of key features that are multi-indicator, personalized, and scientifically sound, as shown in Table 2, was selected for use in subsequent model building.

[0087] S540. Train multiple base learners based on the key feature set, and use ensemble learning and Bayesian optimization to determine the optimal weight combination of the base learners to obtain the fatigue detection model. It is understood that this embodiment of the invention uses an ensemble learning method, combining multiple base learners such as SVM and Random Forest, and determines the optimal weight combination through Bayesian optimization to achieve high-precision detection. Accordingly, the fatigue detection model training process in this step includes: handling outliers in the key feature set; normalizing the outlier-handled key feature set and training each base learner; designing the weight space for each base learner; designing an objective function with negative cross-validation accuracy as the optimization objective; modeling the distribution of the objective function based on a Gaussian process, and using expected improvement to guide the search direction of the weight space to obtain the global optimal solution; normalizing the global optimal solution to ensure that the weights of each base learner are non-negative and sum to 1, constructing the final ensemble classifier, which serves as the trained motion fatigue detection model.

[0088] More specifically, the training process for this fatigue detection model is as follows:

[0089] (1) Handling outliers

[0090] To address the outlier problem in multi-dimensional key feature sets, an ensemble detection method based on isolated forests is employed for systematic cleaning. First, invalid columns with all-zero values ​​are removed. For missing values ​​in numerical features, mean imputation is used to fill in the gaps, ensuring data integrity. To avoid the influence of dimensional differences on the detection results, Z-score is used to normalize the processed data.

[0091] For example, in the anomaly detection phase, a prior parameter of 5% is set for the proportion of anomalous samples in the isolated forest. The isolation level of samples is quantified by constructing 100 random binary trees, and data points with path lengths significantly lower than normal samples are identified as outliers. Finally, detected anomalous samples (e.g., accounting for 5% of the total samples) are removed from the key feature set. This method ensures experimental reproducibility through random seeds and is more suitable for high-dimensional nonlinear data distribution scenarios compared to traditional threshold-based methods.

[0092] (2) Weight selection for ensemble learning models based on Bayesian optimization

[0093] First, the StandardScaler data standardization method can be used to normalize the set of key features after anomaly processing to ensure that the scale of different features is consistent, so as to train each base learner.

[0094] It should be noted that the base learners selected in the embodiments of this invention include heterogeneous models such as Support Vector Machine (SVM), Random Forest, Stacking Classifier, and Extra Trees. Among them, SVM is good at high-dimensional boundary partitioning, Random Forest and Extra Trees handle nonlinear relationships and noise through tree ensemble, Extra Trees adds additional randomness, and Stacking integrates the advantages of each model through meta-learning. The models complement each other in terms of learning methods and decision boundaries.

[0095] Next, the four models are integrated using soft voting, and Bayesian optimization is used to determine the optimal weight combination for the models. The relevant details are as follows:

[0096] 1) Weight search space and constraint modeling

[0097] Design the weight space for each base learner and define the feasible region of the weight parameters (the weights w1, w2, w3 of the first three models ∈ [0, 1], and the fourth weight w4 is automatically filled in). This simplifies the four-dimensional weight optimization into a three-dimensional problem and reduces computational complexity.

[0098] The weights to be optimized are w1, w2, w3 ∈ [0, 1]. The fourth weight is calculated based on the constraints.

[0099] w4 = 1 - w1 - w2 - w3

[0100] Constraints:

[0101] w1+w2+w3≤1,w4≥0

[0102] 2) Objective function design and cross-validation evaluation

[0103] The objective function f(w) is designed with the negative cross-validation accuracy as the optimization goal. Combined with grouped cross-validation to prevent data leakage and accurately evaluate the weight performance.

[0104]

[0105] Wherein, CV_accuracy(w) is the cross-validation accuracy, calculated by the performance of the current weight combination w on the validation set; an invalid weight combination returns 10. 6 Invalid solutions are forcibly excluded.

[0106] 3) Bayesian optimization iterative search for optimal weights

[0107] Based on a Gaussian process surrogate model and an expected improved acquisition function, this method efficiently explores the weight space, balances local development with global exploration, and uses a probabilistic model to guide the search direction, approximating / obtaining the global optimum with the fewest evaluations. Specifically:

[0108] Proxy model:

[0109] Gaussian Process (GP) modeling objective function distribution:

[0110] f(w)~GP(0,k RBF (w,w′))

[0111] The RBF kernel function is:

[0112]

[0113] The RBF kernel function measures the similarity between two weight combinations w and w′, exp is the natural exponential function, and the length scale l controls the smoothness of the kernel function.

[0114] Acquisition function:

[0115] Desired improvements to guide search direction:

[0116]

[0117] Among them, w t+1 The optimal weight combination selected for the next iteration. To calculate the expected value, f min This is the currently known optimal target value.

[0118] 4) Weight normalization and ensemble model construction

[0119] The global optimal solution is normalized to ensure that the weights of each base learner are non-negative and sum to 1. This normalization process is then used to construct the final soft-voting ensemble classifier, which serves as the trained motion fatigue detection model. Ensuring that all weights are non-negative and sum to 1 is expressed as follows:

[0120]

[0121] Thus, this embodiment of the invention completes the construction of the soft voting integrated classification model, which can then be used for non-contact detection of motion fatigue.

[0122] It should also be noted that the autonomic regulation of the heart changes during exercise fatigue. Due to decreased parasympathetic activity and increased sympathetic activity, heart rate variability typically decreases. Head posture may droop or tilt, eye focus may become unfocused, and frequent blinking may occur. Therefore, by summarizing existing literature on exercise fatigue, it is clear that heart rate characteristics and AU25 in facial behavior are significantly related to exercise fatigue. Based on visible light video, corresponding rPPG signals and target behavioral indicator sequences can be extracted. rPPG signals and target behavioral indicator sequences can characterize fatigue states. When training a fatigue detection model, data is input from both physiological and behavioral indicators.

[0123] This application addresses the detection of chronic exercise fatigue caused by the cumulative effects of high-intensity or long-term exercise. It incorporates data from a second resting state during detection, comprehensively considering both the general physiological responses to exercise fatigue and individual differences to achieve personalized exercise fatigue detection. By calculating the difference between the first and second resting states, a personalized combination of exercise fatigue characteristics is constructed, fully considering individual differences and significantly improving the targeting of the detection. This application focuses on the chronic exercise fatigue state induced by the cumulative effects of high-intensity or long-term exercise, providing a technical foundation for more comprehensive fatigue management.

[0124] In one example, the aforementioned physiological characteristics include heart rate (HR) and heart rate variability (HRV), where HRV is one or a combination of the following characteristics:

[0125] Temporal characteristics: NN interval, NN interval standard deviation, percentage of adjacent NN interval differences >50ms, mean NN interval, root mean square of adjacent NN interval differences; where NN represents a normal sinus beat;

[0126] Frequency domain characteristics: low-frequency power, high-frequency power, and the power ratio of low-frequency to high-frequency frequencies;

[0127] Nonlinear characteristics: instantaneous heart rate variability standard deviation, long-term heart rate variability standard deviation, and the ratio of instantaneous heart rate variability standard deviation to long-term heart rate variability standard deviation.

[0128] In some embodiments, the training process of the psychological stress detection model includes:

[0129] S610. Construct a model framework including an rPPG variational peak detector, an EDA variational autoencoder, a multimodal encoder, and a variational instance adaptive classifier; the rPPG variational peak detector includes a first rPPG encoder and a peak detector, the EDA variational autoencoder includes a first EDA encoder and an EDA decoder, the multimodal encoder includes a second rPPG encoder and a second EDA encoder, and the variational instance adaptive classifier includes a first shared linear layer, an adaptive module, and a second shared linear layer;

[0130] S620. Obtain rPPG signal and EDA signal for model framework training; use rPPG signal as input to first rPPG encoder to learn first latent variable, and estimate peak position through peak detector to construct first ELBO loss; and use EDA signal as input to first EDA encoder to learn second latent variable, and obtain reconstructed EDA signal through EDA decoder to construct second ELBO loss.

[0131] S630. The rPPG signal and EDA signal are used as the inputs of the second rPPG encoder and the second EDA encoder, respectively. The rPPG features and EDA features are obtained respectively. After concatenating the rPPG features and EDA features, multimodal fusion features are obtained through a linear layer and a LeakyReLU activation layer. The rPPG features and EDA features are divided into rPPG shared features and rPPG specific features, as well as EDA shared features and EDA specific features. A multimodal supervised contrast loss is constructed based on the rPPG shared features and EDA shared features. A distance loss is constructed based on the shared features and specific features within each modality. A cross-modal conversion loss is constructed based on the rPPG shared features and EDA shared features.

[0132] S640. The first latent variable and the second latent variable are used as inputs to the adaptive module to obtain individual-specific adaptive parameters. The multimodal fusion features are used as inputs to the first shared linear layer. The adaptive parameters are combined with the stress recognition results obtained through the second shared linear layer to construct the classification loss.

[0133] S650, based on the first ELBO loss, the second ELBO loss, the multimodal supervised contrast loss, the distance loss, the cross-modal transformation loss and the classification loss, constructs a joint loss, trains the model framework until convergence, and obtains the psychological stress detection model.

[0134] In the embodiments of this application, it is understood that a multimodal domain generalization strategy is integrated into the multimodal encoder, giving the features extracted by the encoder certain domain-invariant properties, making it easier to distinguish the stress and non-stress state features of different individuals. A variational instance adaptation strategy is integrated into the classifier. This strategy utilizes variational inference, approximating the actual distribution of an individual's physiological characteristics through variational distributions using instances of the individual's physiological signals. Latent variables sampled from the variational distribution are used as the basis for classifier weight adaptation, enabling the model to generate decision boundaries suitable for the current individual when facing different individuals. This application, by combining multimodal domain generalization with variational instance adaptation to adjust the decision boundary, can improve the performance of individual stress identification under distribution shift conditions.

[0135] In some embodiments, this application provides a non-contact pressure and fatigue state identification system 700 based on dual-signal joint reasoning, such as... Figure 3 As shown, the non-contact pressure and fatigue state identification system 700 based on dual-indicator joint reasoning may include the following modules:

[0136] Acquisition module 710 is used to acquire visible light video of the face through a camera and to acquire a first EDA signal through a wrist-worn wearable device;

[0137] The extraction module 720 is used to extract rPPG signals and a first facial behavioral indicator sequence based on facial visible light video; the rPPG signals and the first EDA signals are physiological indicator information.

[0138] Alignment module 730 is used to resample and sort the signal amplitudes of the first EDA signal and the first facial behavior indicator sequence to obtain an aligned second EDA signal and a second facial behavior indicator sequence.

[0139] The weighting module 740 is used to perform cross-weighting processing on the second EDA signal and the second facial behavior indicator sequence to obtain the target EDA signal and the target behavior indicator sequence;

[0140] The fatigue detection module 750 is used to input rPPG signals and target behavior indicator sequences into a preset fatigue detection model to obtain fatigue information; the fatigue detection model is an integrated classification model for non-contact detection of motion fatigue.

[0141] The psychological stress detection module 760 is used to input the rPPG signal and the target EDA signal into a preset psychological stress detection model to obtain psychological stress information;

[0142] The synchronous output module 770 is used to synchronously output and display fatigue information and psychological stress information based on a preset signal waveform comparison display area.

[0143] According to embodiments of this application, any and multiple modules among the acquisition module 710, extraction module 720, alignment module 730, weighting module 740, fatigue detection module 750, psychological stress detection module 760, and synchronous output module 770 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module.

[0144] Figure 3 Each module in the system shown has the function of implementing each step in the aforementioned non-contact pressure and fatigue state identification method with dual-signal joint reasoning, and can achieve its corresponding technical effect. For the sake of brevity, it will not be elaborated here.

[0145] In some embodiments, this application provides an electronic device, the structural schematic of which is shown below. Figure 4 As shown.

[0146] The electronic device may include a processor 810 and a memory 820 storing computer program instructions.

[0147] Specifically, the processor 810 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0148] Memory 820 may include mass storage for data or instructions. For example, and not limitingly, memory 820 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 820 may include removable or non-removable (or fixed) media. Where appropriate, memory 820 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 820 is non-volatile solid-state memory.

[0149] Memory 820 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory 820 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the dual-signal joint reasoning non-contact pressure and fatigue state identification methods in the above embodiments.

[0150] The processor 810 reads and executes computer program instructions stored in the memory 820 to implement any of the non-contact pressure and fatigue state identification methods of dual-signal joint reasoning in the above embodiments.

[0151] In one example, the electronic device may also include a communication interface 830 and a bus 800. For example, Figure 4 As shown, the processor 810, memory 820, and communication interface 830 are connected via bus 800 and communicate with each other.

[0152] The communication interface 830 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0153] Bus 800 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 800 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0154] Furthermore, in conjunction with the non-contact pressure and fatigue state identification method based on dual-signal joint reasoning in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any one of the non-contact pressure and fatigue state identification methods based on dual-signal joint reasoning in the above embodiments.

[0155] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0156] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0157] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0158] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0159] In summary, compared with the prior art, this application has the following beneficial effects:

[0160] 1. This application aims to jointly analyze psychological stress and fatigue states by acquiring facial visible light video and a first EDA signal, and extracting rPPG signal and a first facial behavior indicator sequence. To address the issues of excessively long signal sequences and noise, the signal amplitudes of the first EDA signal and the first facial behavior indicator sequence are resampled and sorted to achieve signal alignment. Since the rPPG signal and the first facial behavior indicator sequence are already aligned, after the aforementioned resampling and sorting, this application achieves frame rate alignment, distance alignment, and location alignment among the rPPG signal, the second EDA signal, and the second facial behavior indicator sequence.

[0161] 2. Since physiological indicators and behavioral indicators are interrelated, after obtaining multimodal data through signal extraction and alignment, this application performs cross-weighting processing on the second EDA signal and the second facial behavioral indicator sequence. That is, it dynamically weights the intrinsic physiological signals and facial behavioral features to achieve interpretable, efficient and direct cross-modal information interaction without relying on large-scale labeled data for iterative gradient optimization training.

[0162] 3. This application obtains fatigue and psychological stress information through model analysis of signals and outputs them synchronously for comparative analysis. This application simultaneously analyzes psychological stress and fatigue states, using physiological and behavioral indicators for simultaneous joint analysis to identify cross-scenario state fluctuations. This dual-indicator identification improves the detection accuracy and efficiency for psychological stress and fatigue states.

[0163] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A non-contact method for identifying pressure and fatigue states using dual-indicator joint reasoning, characterized in that, include: The camera captures visible light video of the face, and the first EDA signal is acquired through a wrist-worn wearable device. Based on the facial visible light video, rPPG signals and a first facial behavioral indicator sequence are extracted; the rPPG signals and the first EDA signals are physiological indicator information. The signal amplitudes of the first EDA signal and the first facial behavior indicator sequence are resampled and sorted to obtain an aligned second EDA signal and a second facial behavior indicator sequence. The second EDA signal and the second facial behavior indicator sequence are cross-weighted to obtain the target EDA signal and the target behavior indicator sequence. The rPPG signal and the target behavior indicator sequence are input into a preset fatigue detection model to obtain fatigue information; The fatigue detection model is an integrated classification model for non-contact detection of motion fatigue; The rPPG signal and the target EDA signal are input into a preset psychological stress detection model to obtain psychological stress information; The fatigue information and psychological stress information are synchronously output and displayed based on the preset signal waveform comparison display area.

2. The non-contact pressure and fatigue state identification method based on dual-signal joint reasoning as described in claim 1, characterized in that, The step of performing cross-weighted processing on the second EDA signal and the second facial behavior indicator sequence to obtain the target EDA signal and the target behavior indicator sequence includes: The second facial behavior indicator sequence is weighted based on the second EDA signal using a preset weighting strategy, and the target behavior indicator sequence is obtained through cross-modal information interaction. Based on the second facial behavior indicator sequence, the second EDA signal is weighted using the weighting strategy, and the target EDA signal is obtained through cross-modal information interaction.

3. The non-contact pressure and fatigue state identification method based on dual-signal joint reasoning as described in claim 2, characterized in that, The step involves weighting the second facial behavior indicator sequence based on the second EDA signal using a preset weighting strategy, and obtaining the target behavior indicator sequence through cross-modal information interaction, including: Based on the time sequence of the second EDA signal, a scalar index sequence M(t) is determined to measure the individual's physiological arousal level in real time; the scalar index sequence M(t) is the physiological signal energy, the number of peaks, or a more specific measure with clear physiological significance within a time window; The scalar index sequence M(t) is transformed into a time-varying attention weight sequence w(t) through a pre-defined, non-linear deterministic attention weight mapping function f; wherein, the deterministic attention weight mapping function f is used to map the absolute value or relative change of the physiological index to a weight interval normalized to [0,1]. Using the attention weight sequence w(t), a multiplicative gain control or element-wise weighting operation is performed on the synchronously acquired second facial behavior indicator sequence to generate the target behavior indicator sequence.

4. The non-contact pressure and fatigue state identification method based on dual-signal joint reasoning as described in claim 3, characterized in that, The calculation process of the attention weight sequence w(t) satisfies the expression: Where μ is a preset physiological index threshold, and σ is a scaling factor that controls the sharpness of the transition. The calculation process of the target behavior indicator sequence satisfies the expression: AT enhanced (t)=G(w(t))·AU(t) Among them, AU enhanced (t) represents the target behavior indicator sequence, G(w(t)) is a gain function with w(t) as input, G(w(t))=1+α·w(t), α is the preset modulation intensity coefficient, and AU(t) is the second facial behavior indicator sequence.

5. The non-contact pressure and fatigue state identification method based on dual-signal joint reasoning as described in any one of claims 1-4, characterized in that, The step of resampling and sorting the signal amplitudes of the first EDA signal and the first facial behavior indicator sequence to obtain an aligned second EDA signal and a second facial behavior indicator sequence includes: A target synchronization frequency is preset, and a standard, uniformly distributed target time vector is generated based on the start and end timestamps of the signal segment to be processed, which serves as a unified time reference for the synchronization of all signals; wherein, the first EDA signal and the first facial behavior indicator sequence are both signal segments to be processed. Using the timestamp sequence of the first EDA signal and the first facial behavior indicator sequence as independent variables and the amplitude sequence as dependent variables, a continuous interpolation function is constructed; the type of the interpolation function is selected according to the signal characteristics and processing accuracy requirements, and it can estimate the signal amplitude at any time based on discrete raw data points; The interpolation function is used to evaluate the target time vector at each time point to calculate the resampled signal amplitude of the first EDA signal and the first facial behavior indicator sequence. Arrange all the calculated signal amplitudes in order, and determine the two discrete-time signal sequences after resampling by arranging them to obtain the second EDA signal and the second facial behavior indicator sequence.

6. The non-contact pressure and fatigue state identification method based on dual-signal joint reasoning as described in any one of claims 1-4, characterized in that, The training process of the fatigue detection model includes: First and second facial visible light videos corresponding to the first and second resting states are acquired respectively, and corresponding physiological features and facial behavioral features are extracted. The physiological features and facial behavior features corresponding to the first resting state are used as baseline features, the physiological features and facial behavior features corresponding to the second resting state are used as instantaneous state features, and the difference between the instantaneous state features and the baseline features is used as the baseline relative features. Based on domain knowledge and statistical tests, feature selection is performed on the set of motion fatigue features, including the instantaneous state features and the baseline relative features, to obtain a key feature set. Multiple base learners are trained based on the set of key features. The optimal weight combination of the base learners is determined by ensemble learning and Bayesian optimization to obtain the fatigue detection model.

7. The non-contact pressure and fatigue state identification method based on dual-signal joint reasoning as described in any one of claims 1-4, characterized in that, The first facial behavior indicator sequence includes a head posture sequence, an eye gaze angle sequence, and a facial behavior unit feature sequence; The head posture sequence describes the head's motion state through translation and rotation in three-dimensional space; the eye gaze angle sequence describes the overall change in eye gaze direction by capturing the displacement of the eyes in the horizontal and vertical directions, combined with horizontal and vertical deflection. The facial behavior unit feature sequence corresponds to the following actions: raising the inner eyebrow, raising the outer eyebrow, lowering the eyebrow, raising the upper eyelid, raising the cheek, tightening the eyelid, wrinkling the nose, raising the upper lip, pulling up the corner of the mouth, tightening the corner of the mouth, lowering the corner of the mouth, raising the chin, stretching the lips, tightening the lips, opening the lips, opening the jaw, sucking the lips, and blinking.

8. A non-contact pressure and fatigue state identification system based on dual-signature joint reasoning, characterized in that, include: The acquisition module is used to acquire visible light video of the face through a camera and to acquire the first EDA signal through a wrist-worn wearable device; The extraction module is used to extract rPPG signals and a first facial behavioral indicator sequence based on the facial visible light video; the rPPG signals and the first EDA signals are physiological indicator information. The alignment module is used to resample and sort the signal amplitudes of the first EDA signal and the first facial behavior indicator sequence to obtain an aligned second EDA signal and a second facial behavior indicator sequence. The weighting module is used to perform cross-weighting processing on the second EDA signal and the second facial behavior indicator sequence to obtain the target EDA signal and the target behavior indicator sequence; The fatigue detection module is used to input the rPPG signal and the target behavior indicator sequence into a preset fatigue detection model to obtain fatigue information; The fatigue detection model is an integrated classification model for non-contact detection of motion fatigue; The psychological stress detection module is used to input the rPPG signal and the target EDA signal into a preset psychological stress detection model to obtain psychological stress information; The synchronous output module is used to synchronously output and display the fatigue information and the psychological stress information based on a preset signal waveform comparison display area.

9. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements a non-contact pressure and fatigue state identification method based on dual-signal joint reasoning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that, when executed by a processor, implement the non-contact pressure and fatigue state identification method according to any one of claims 1 to 7, based on dual-signal joint reasoning.

Images