A method for assessing sleep quality based on multimodal features during wakefulness

Abstract

This invention discloses a method for assessing sleep quality based on multimodal features in a waking state, comprising: S1, guiding subjects to complete a sleep quality scale, collecting pulse wave signals, facial thermal images, and corresponding temperature matrices of subjects in a resting state, and conducting a psychological alertness task test; S2, extracting heart rate, heart rate variability, and pulse wave morphology features from the pulse wave signals; S3, identifying key points on the face, determining regions of interest, and extracting facial temperature features; S4, analyzing the acquired psychological alertness task test data, statistically analyzing and extracting reaction time and attention maintenance-related indicators; S5, constructing a training dataset; S6, using the training dataset to train a classifier and construct a sleep quality classification model; S7, inputting the multimodal features collected from new users in a waking state into the sleep quality classification model and outputting the assessment results. This invention provides a method for assessing sleep quality in a daytime waking state.

Description

Technical Field

[0001] This invention belongs to the field of sleep quality assessment technology, specifically relating to a sleep quality assessment method based on multimodal characteristics in a waking state. Background Technology

[0002] Currently, mainstream sleep quality assessment methods mainly fall into two categories: subjective scale assessment and objective physiological monitoring. Subjective scale assessment methods are primarily represented by the Pittsburgh Sleep Quality Index (PSQI) questionnaire. These methods rely heavily on the subject's self-recall and subjective judgment, making them susceptible to memory bias and subjective emotions, and thus difficult to accurately and objectively reflect an individual's sleep status. Objective physiological monitoring, on the other hand, often employs polysomnography (PSG), which continuously records the nighttime sleep process using various physiological signals such as electroencephalography (EEG), electrooculography (EOG), and electromyography (EMG). Although PSG is considered the gold standard for sleep assessment, its equipment is complex and costly, and the monitoring process is highly invasive, making it difficult to promote and apply in daily life or large-scale populations.

[0003] In recent years, with the development of wearable devices, methods for assessing sleep quality based on signals such as heart rate variability and pulse waves have gradually attracted attention. However, existing methods mostly focus on long-term monitoring and assessment of physiological signals during nighttime sleep, and still rely on wearable or contact devices, which are difficult to meet the actual needs for non-intrusive and low-interference monitoring of the sleep process.

[0004] Current research has proposed utilizing the concept of the "post-sleep effect," which involves inverting nighttime sleep-related states through physiological representations during daytime wakefulness. For example, pulse wave signals are collected under waking and resting conditions, and a mapping relationship is established between their characteristics and the risk of nighttime snoring. This type of work validates the feasibility of using waking state characteristics for sleep-related assessments. However, existing techniques are mostly geared towards the classification and prediction of single sleep-related risks, lacking a systematic assessment scheme that directly maps multimodal objective characteristics during daytime wakefulness into standardized sleep quality levels or scores. Furthermore, there is still room for improvement in cross-modal feature fusion and cross-individual robustness. Summary of the Invention

[0005] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and propose a sleep quality assessment method based on multimodal characteristics in a waking state. This method comprehensively assesses sleep quality by acquiring pulse wave physiological characteristics, facial infrared temperature characteristics, and psychomotor alertness task behavior characteristics during the daytime waking state.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for assessing sleep quality based on multimodal characteristics during wakefulness includes the following steps:

[0008] S1. Construct a test scenario, guide the subjects to fill in a sleep quality scale, collect pulse wave signals, facial thermal imaging images and corresponding temperature matrices of the subjects in a resting state, conduct psychological alertness task tests on the subjects and record the test data.

[0009] S2. Preprocess the collected pulse wave signal to extract heart rate, heart rate variability and pulse wave morphology features.

[0010] S3. Preprocess the acquired facial thermal imaging images and corresponding temperature matrix, identify key points on the face, determine the region of interest, and extract facial temperature features.

[0011] S4. Analyze the acquired psychological alertness task test data, and statistically extract indicators related to reaction time and attention maintenance.

[0012] S5. Using the physiological and behavioral characteristics of the subjects when they are awake as input and the sleep quality scale scores as sample labels, construct a training dataset.

[0013] S6. Use the training dataset to train the classifier and build a sleep quality classification model;

[0014] S7. Input the multimodal features collected from the new user in a waking state into the sleep quality classification model and output the sleep quality assessment results.

[0015] Furthermore, step S1 specifically includes:

[0016] S11. Construct a test scenario; Under suitable temperature and ventilation conditions, guide the subjects to fill in a sleep quality scale, obtain the scale score according to the pre-designed scoring rules of the scale, and generate a sample label Label according to the preset threshold T. After the subjects have filled in the scale, guide them to sit quietly for 5 minutes as a rest adaptation period to allow them to enter a relaxed state.

[0017] S12. Collect pulse wave signals, facial thermal images, and corresponding temperature matrices of the subject in a resting state.

[0018] S13. After the subject completes the collection of physiological signals, guide the subject to complete the psychological alertness task PVT test and record the raw PVT behavioral data.

[0019] Furthermore, step S2 specifically includes:

[0020] S21. The acquired pulse wave signal is preprocessed, including polarity normalization, high-frequency noise suppression, and baseline drift correction. Specifically, the acquired pulse wave signal is first normalized to ensure that the main peak is upward. Then, a preset order FIR low-pass filter is used to filter the signal to suppress high-frequency noise generated by environmental electromagnetic fields and electromyography. Finally, the low-frequency trend term of the signal is estimated using median filtering to estimate the baseline. The baseline drift caused by respiration and body movement is eliminated by subtracting the baseline estimation result from the low-pass filtered pulse wave signal.

[0021] S22. Perform periodic segmentation on the preprocessed pulse wave signal, extract the pulse wave interval sequence between adjacent pulse wave periods, evaluate the effectiveness of the pulse wave period, and remove abnormal periods or pulse wave periods with poor quality.

[0022] S23. Construct a pulse wave interval sequence based on the retained effective pulse wave period, and extract the time-domain and frequency-domain features of pulse wave variability based on the pulse wave interval sequence;

[0023] S24. Based on the retained effective pulse wave cycle, analyze the morphological structure of a single pulse wave cycle and extract the morphological characteristic parameters of the pulse wave.

[0024] The pulse wave features extracted in steps S23 and S24 constitute a pulse wave feature set. .

[0025] Further, in step S22, based on the preprocessed pulse wave signal, the position of the main peak of the pulse wave is first detected, and the corresponding pulse wave foot point is located within a preset time window before each main peak, where the foot point serves as the boundary point of the pulse wave period; after periodically segmenting the pulse wave signal, a pulse wave interval sequence is constructed; then, the segmented pulse wave signal is subjected to morphological consistency screening, by performing correlation analysis between the morphology of a single-cycle pulse wave signal and a reference template, eliminating cycles with obvious morphological distortion; wherein, the reference template is obtained by the following steps:

[0026] In the set of effective pulse cycles that meet the physiological rationality constraints, each pulse cycle is resampled to a uniform length and its amplitude is standardized. Then, the normalized waveforms of each cycle are statistically analyzed point by point, and the median is taken to form a representative cycle waveform. Using this representative cycle waveform as a reference template, the correlation coefficient between any cycle to be evaluated and the reference template is calculated. Cycles with a correlation coefficient lower than a preset threshold are judged as morphological abnormalities and are removed.

[0027] In step S23, based on the removal of distorted cycles, the remaining pulse wave interval sequence is subjected to cubic spline interpolation resampling to compensate for the discontinuity caused by the removal of abnormal cycles, and a pulse wave interval variation sequence with equal time intervals is obtained. Based on this sequence, indicators reflecting the characteristics of autonomic nervous activity are calculated, including time domain features and frequency domain features.

[0028] In step S24, based on the single-cycle waveform that has passed the effectiveness assessment, the main peak, diphtheria notch, and diphtheria peak are identified using the first and second derivatives. By calculating the time-domain coordinates and amplitudes of the above feature points, morphological parameters reflecting the cardiovascular state are extracted, including rise / fall characteristics and amplitude characteristics.

[0029] Furthermore, step S3 specifically includes:

[0030] S31. Preprocess the acquired facial thermal imaging images and corresponding temperature matrices, including removing invalid data frames and aligning timestamps. Invalid data frames include key point occlusion, missing temperature matrix, and missing facial images.

[0031] S32. Use a key point detection model to identify facial key points and obtain the coordinates of key points on the forehead, around the eyes, and the tip of the nose; based on the geometric relationship of the key points and adaptive constraints on face size, determine the coordinate range of the region of interest (ROI).

[0032] S33. Based on the determined regions of interest (ROIs), extract the temperature values ​​of the pixels within the corresponding temperature matrix, and construct the temperature time series of each region of interest according to the acquisition time order; extract facial temperature features that characterize the temperature level and temporal fluctuation characteristics of local facial regions. The facial temperature features include temperature level statistical features, temperature dispersion features, temporal dynamic features, and regional symmetry and regional difference features.

[0033] The extracted facial temperature features are used to construct a facial temperature feature set. .

[0034] Furthermore, in step S32, a key point detection model is used to infer the key points of the forehead in each frame of the facial thermal imaging image. Key points around the right eye Key points around the left eye and key points of the tip of the nose Pixel coordinates;

[0035] The interocular distance is calculated as a facial scale parameter based on the coordinates of key points around the left and right eyes. The calculation formula is as follows:

[0036] ;

[0037] Centered on any keypoint P(x,y), based on the face scale parameter d eye The width and height of the ROI are determined proportionally, and a vertical offset is introduced to adapt to different facial areas; the formula for calculating the ROI boundary coordinates is:

[0038] ;

[0039] ;

[0040] ;

[0041] ;

[0042] in, This is the ROI width scaling factor. This is the ROI height ratio coefficient. This is the vertical offset scaling factor.

[0043] Furthermore, step S4 specifically involves:

[0044] Based on the raw PVT behavioral data, response behavior data were obtained, and behavioral feature parameters characterizing the subjects' alertness level and attention maintenance ability were extracted. The extracted behavioral feature parameters constituted the PVT behavioral feature set. .

[0045] Furthermore, step S5 specifically involves:

[0046] pulse wave feature set Facial temperature feature set and PVT behavioral feature set The dataset is constructed by fusing the scores of a sleep quality scale with preset thresholds as sample labels.

[0047] For subjects with poor sleep quality, the sample composition is as follows: Input features Sample Labels ;

[0048] For subjects with good sleep quality, the sample composition is as follows: Input features Sample Labels .

[0049] Furthermore, step S6 specifically includes:

[0050] S61. Use the dataset constructed in step S5 as the sample training set. During the model training process, use K-fold cross-validation to divide the training sample set into multiple subsets. During the training process, select a subset as the validation set in turn, and use the remaining subset as the training set for the training and validation of the classifier.

[0051] S62. Based on the verification partitioning results of step S61, the classifier is trained using a random forest binary classification model and verified on the validation set, thereby learning the mapping relationship between input features and sample labels.

[0052] S63. Based on the training and validation results, comprehensively evaluate the classifier performance and determine the final sleep quality classification model.

[0053] Furthermore, step S7 specifically includes:

[0054] S71. For test subjects who are unaware of their own sleep quality, collect the user's pulse physiological behavior data according to step S1.

[0055] S72. Process the user's physiological behavior data according to steps S2 to S5 to obtain the user's input features. ;

[0056] S73. Input the user's input features into the sleep quality classification model to assess the user's nighttime sleep quality while the user is awake during the day.

[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0058] 1. This invention integrates pulse wave signals, facial infrared thermal imaging data, and PVT behavioral data to perform multimodal joint modeling of sleep quality in subjects. Pulse wave features characterize sleep-related autonomic nervous system regulation and cardiovascular status changes, while facial thermal imaging features are introduced to supplement peripheral vasomotor and surface thermoregulation information. PVT behavioral features are also introduced to characterize the functional aftereffects of sleep deprivation on daytime alertness and attention maintenance. The three form a complementary evidence chain of "physiological regulation-thermal regulation-behavioral performance," which can effectively reduce the influence of individual differences, wearing status, and transient noise compared to relying on a single physiological modality, thereby improving the stability, interpretability, and robustness of sleep quality assessment.

[0059] 2. This invention uses sleep quality scale scores as supervisory labels to establish a quantitative mapping model, which maps the objective physiological and behavioral characteristics collected by the subjects in a waking state into standardized sleep quality levels. This allows for quantifiable outputs without the need for subjects to fill out a scale during actual use, thereby reducing the uncertainty caused by subjective recall bias and individual subjective differences, and improving the repeatability and cross-comparability of the assessment results.

[0060] 3. This invention completes the collection and analysis of physiological signals and behavioral data during the daytime waking state, eliminating the need for long-term nighttime monitoring or polysomnography equipment, thus reducing the complexity of the testing process and the burden on the subjects, and improving the feasibility and applicability of the method in practical application scenarios. Attached Figure Description

[0061] Figure 1 This is a flowchart of the method of the present invention.

[0062] Figure 2 This is the PVT test interface in the embodiment.

[0063] Figure 3 This is a sleep quality scale revised based on the Pittsburgh Sleep Quality Index in the embodiment.

[0064] Figure 4 This is a flowchart of pulse wave signal preprocessing and physiological index extraction in the embodiment.

[0065] Figure 5 This is the effect of pulse wave waveform preprocessing in the embodiment.

[0066] Figure 6 This is a flowchart of facial thermal imaging image preprocessing and physiological indicator extraction in this embodiment.

[0067] Figure 7 This is a ROI segmentation map of the facial thermal imaging image in the embodiment. Detailed Implementation

[0068] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0069] Examples; such as Figure 1 As shown, a sleep quality assessment method based on multimodal features in a waking state includes the following steps:

[0070] S1. Construct a test scenario, collect pulse wave signals, facial thermal imaging images and corresponding temperature matrices of the test subjects in a resting state, conduct psychological alertness task tests on the subjects and record the test data.

[0071] In this embodiment, the testing equipment consists of a pulse wave signal measurement sensor, an infrared thermal imaging camera, and a supporting host computer.

[0072] Step S1 is as follows:

[0073] Under suitable temperature and ventilation conditions, participants were guided to complete a sleep quality scale. The scale's pre-set scoring rules yielded scores, which were then used to determine the scores based on pre-set thresholds. A sample label is generated. After completing the questionnaire, participants are guided to sit quietly for 5 minutes as a resting adaptation period to allow them to relax. Subsequently, pulse wave signals, facial thermal images, and corresponding temperature matrix data are collected while the participants remain at rest. Finally, participants take a Psychomotor Vigilance Task (PVT) test, and their reaction time is recorded. The PVT test interface is shown below. Figure 2 As shown, the system presents a target stimulus (e.g., a red dot) at a designated location on the screen after a random or preset time interval. When the subject observes the appearance of the target stimulus, they need to trigger a preset key (e.g., the space bar) as soon as possible. The host computer records the time difference from the time the target stimulus is presented to the time the key is triggered as the reaction time for that instance, and summarizes and stores the reaction times for each instance.

[0074] The sleep quality scale was revised based on the Pittsburgh Sleep Quality Index (PSQI) scoring framework, adjusting the time scale to reflect the recall of the previous night's sleep quality. To accommodate the student population as the subjects, items related to sleep disturbances such as noise or roommate interference were added. The sleep quality scale includes... Figure 3 As shown.

[0075] In this embodiment, the acquisition time for both the pulse wave signal and the facial thermal imaging data is uniformly set to 120s; wherein, the sampling frequency of the pulse wave signal is set to 400Hz, and the sampling frequency of the facial thermal imaging image is set to 5Hz.

[0076] S2. Preprocess the acquired pulse wave signal to extract heart rate, heart rate variability, and pulse wave morphology features; in this embodiment, such as... Figure 4 As shown, it includes:

[0077] S21. A 12Hz FIR low-pass filter is used to remove high-frequency noise from the acquired pulse wave signal, resulting in a smoothed pulse wave signal. Subsequently, median filtering is applied to the smoothed pulse wave signal to estimate the signal baseline, and baseline drift is eliminated by subtracting the baseline estimation result from the original signal. The pulse wave preprocessing effect is as follows: Figure 5 As shown.

[0078] S22. Locate the main peak of the pulse wave by finding local maxima. Based on the position of the main peak, search a time window of 140 sampling points forward to find the minimum point as the foot point of the pulse wave cycle to segment the pulse wave cycle. Then, evaluate the effectiveness of the segmented pulse wave cycle, including:

[0079] 1) Interval constraint between adjacent points: Statistically count the time interval between adjacent points (IBI) and remove cycles with a duration of less than 0.33s or greater than 1.50s;

[0080] 2) Jump detection: Using the mean interval of a moving median window of length 7, abnormal cycles that deviate from the mean by more than 20% are removed;

[0081] 3) Morphological Screening: The median of all candidate periods within the segment is selected to generate a reference template waveform. Each single period is resampled to a length of 200 points and standardized using Robust Z-score. The Pearson correlation coefficient between the single period and the reference template is then calculated. If the correlation coefficient is below 0.85, the period is considered to have significant morphological distortion and is discarded.

[0082] S23. Based on the removal of aberrant periods, the remaining pulse wave interval sequence is subjected to cubic spline interpolation resampling to compensate for the discontinuities caused by the removal of abnormal periods, obtaining a pulse wave interval variation sequence with equal time intervals. Based on this sequence, indices reflecting the characteristics of autonomic nervous activity are calculated, including but not limited to:

[0083] Temporal features, such as mean interval, standard deviation of interval (SDNN), difference between adjacent intervals (RMSSD), PNN50, PNN20, etc.;

[0084] Frequency domain characteristics, low-frequency power (LF), high-frequency power (HF), and the ratio of high to low-frequency power (LF / HF).

[0085] S24. Based on the single-cycle waveform that has passed the effectiveness assessment, the main peak, dicrotic notch, and dicrotic peak are identified using the first and second derivatives; by calculating the time-domain coordinates and amplitudes of the above feature points, morphological parameters reflecting the cardiovascular state are extracted, including but not limited to:

[0086] Rising / falling characteristics, rising time (RT), falling time (DT);

[0087] Amplitude characteristics, main wave amplitude, diphtheria wave amplitude.

[0088] The pulse wave features extracted in steps S23 and S24 constitute a pulse wave feature set. .

[0089] S3. Preprocess the acquired facial thermal imaging image and corresponding temperature matrix, identify key points on the face, determine the Region of Interest (ROI), and extract facial temperature features; in this embodiment, such as Figure 6 As shown, it specifically includes:

[0090] S31. Preprocess the acquired facial thermal imaging images and corresponding temperature matrices, including removing invalid data frames and aligning timestamps. Invalid data frames include key point occlusion, missing temperature matrix, and missing facial images.

[0091] S32. Use a key point detection model to infer the key points of the forehead in each frame of facial thermal imaging image. Key points around the right eye Key points around the left eye and key points of the tip of the nose The pixel coordinates; in this embodiment, the key point detection model adopts the YOLOv8-pose pose key point detection network, and can be obtained by acquiring multiple frames of facial thermal imaging images as training samples, annotating the samples with key points and training.

[0092] The interocular distance is calculated as a facial scale parameter based on the coordinates of key points around the left and right eyes. The calculation formula is as follows:

[0093] ;

[0094] Centered on any keypoint P(x,y), based on the face scale parameter d eye The width and height of the ROI are determined proportionally, and a vertical offset is introduced to adapt to different facial areas; the formula for calculating the ROI boundary coordinates is:

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] in, This is the ROI width scaling factor. This is the ROI height ratio coefficient. This represents the vertical offset scaling factor. Different factor values ​​can be set for the forehead, periorbital, and nasal regions. The ROI segmentation effect for key regions is shown below. Figure 7 As shown.

[0100] S33. Based on the determined regions of interest (ROIs), extract the temperature values ​​of pixels within the corresponding temperature matrix, and construct a temperature time series for each ROI according to the acquisition time sequence; extract facial temperature features characterizing the temperature level and temporal fluctuation characteristics of local facial regions, including but not limited to:

[0101] Temperature level statistical characteristics, mean, median, and quantiles of the region of interest (ROI);

[0102] Temperature dispersion characteristics, standard deviation and interquartile range of the region of interest (ROI);

[0103] Temporal dynamic characteristics, time series standard deviation of the region of interest (ROI);

[0104] Regional symmetry and regional differences, temperature asymmetry around the eyes, temperature asymmetry between the left and right foreheads, and temperature difference between the forehead and nose / around the eyes and nose.

[0105] The extracted facial temperature features are used to construct a facial temperature feature set. .

[0106] S4. Analyze the acquired psychological alertness task test data, and statistically extract indicators related to reaction time and attention maintenance; specifically:

[0107] Based on the raw PVT behavioral data, response behavior data were obtained, and behavioral feature parameters characterizing the subjects' alertness level and attention maintenance ability were extracted. The extracted behavioral feature parameters constituted the PVT behavioral feature set. Behavioral characteristic parameters are used to characterize the subjects' alertness level and attention maintenance ability, including average reaction speed, lapse rate, reaction time standard deviation, and the slowest 10% of reaction speed, which are used as input for subsequent sleep quality classification models; where lapse rate refers to... .

[0108] S5. Using the physiological and behavioral characteristics of the subjects when awake as input and the sleep quality scale score as sample label, construct a training dataset; in this embodiment, this specifically includes:

[0109] pulse wave feature set Facial temperature feature set and PVT behavioral feature set The dataset is constructed by fusing the scores of a sleep quality scale with preset thresholds as sample labels.

[0110] Among them, when the subject's sleep quality scale score meets At that time, the subject's sample was labeled as a sample with poor sleep quality, and its sample composition was as follows: Input features Sample Labels ;

[0111] When the subject's sleep quality scale score meets At that time, the subject's sample was labeled as having good sleep quality, and its sample composition was as follows: Input features Sample Labels .

[0112] Wherein, the threshold T is a preset threshold or an adaptive threshold determined based on the sleep quality score distribution of the training samples. In this embodiment, the threshold T=5.

[0113] S6. Train the classifier using the training dataset to build a sleep quality classification model; in this embodiment, this specifically includes:

[0114] S61. Use the dataset constructed in step S5 as the sample training set. During the model training process, use the 5-fold cross-validation method to divide the training sample set into multiple subsets. During the training process, select a subset as the validation set in turn, and use the remaining subset as the training set for the training and validation of the random forest classifier.

[0115] S62. Train a random forest binary classifier on the training subset to complete parameter learning, and validate it on the validation subset to establish a mapping relationship between input features and sleep quality labels.

[0116] S63. In each round of validation, the classification performance index is calculated based on the validation subset, and the results of each round are summarized to comprehensively evaluate the model performance.

[0117] S7. Input the multimodal features collected from the new user's awake state into the sleep quality classification model, and output the sleep quality assessment results; in this embodiment, this specifically includes:

[0118] S71. For test subjects who do not know their own sleep quality, according to step S1, collect the user's physiological behavior data. Unlike the test subjects, the user does not need to fill in a sleep quality questionnaire.

[0119] S72. Process the user's physiological behavior data according to steps S2 to S5 to obtain the user's input features. ;

[0120] S73. Input the user's input features into the sleep quality classification model to assess the user's nighttime sleep quality while the user is awake during the day.

[0121] It should also be noted that, in this specification, terms such as "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0122] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for assessing sleep quality based on multimodal features during wakefulness, characterized in that, Includes the following steps: S1. Construct a test scenario, guide the subjects to fill in a sleep quality scale, collect pulse wave signals, facial thermal imaging images and corresponding temperature matrices of the subjects in a resting state, conduct psychological alertness task tests on the subjects and record the test data. S2. Preprocess the collected pulse wave signal to extract heart rate, heart rate variability and pulse wave morphology features. S3. Preprocess the acquired facial thermal imaging images and corresponding temperature matrix, identify key points on the face, determine the region of interest, and extract facial temperature features. S4. Analyze the acquired psychological alertness task test data, and statistically extract indicators related to reaction time and attention maintenance. S5. Using the physiological and behavioral characteristics of the subjects when they are awake as input and the sleep quality scale scores as sample labels, construct a training dataset. S6. Use the training dataset to train the classifier and build a sleep quality classification model; S7. Input the multimodal features collected from the new user in a waking state into the sleep quality classification model and output the sleep quality assessment results.

2. The sleep quality assessment method based on multimodal features in a waking state according to claim 1, characterized in that, Step S1 specifically includes: S11. Construct a test scenario; Under suitable temperature and ventilation conditions, guide the subjects to fill in a sleep quality scale, obtain the scale score according to the pre-designed scoring rules of the scale, and generate a sample label Label according to the preset threshold T. After the subjects have filled in the scale, guide them to sit quietly for 5 minutes as a rest adaptation period to allow them to enter a relaxed state. S12. Collect pulse wave signals, facial thermal images, and corresponding temperature matrices of the subject in a resting state. S13. After the subject completes the collection of physiological signals, guide the subject to complete the psychological alertness task PVT test and record the raw PVT behavioral data.

3. The sleep quality assessment method based on multimodal features in a waking state according to claim 2, characterized in that, Step S2 specifically includes: S21. The acquired pulse wave signal is preprocessed, including polarity normalization, high-frequency noise suppression, and baseline drift correction. Specifically, the acquired pulse wave signal is first normalized to ensure that the main peak is upward. Then, a preset order FIR low-pass filter is used to filter the signal to suppress high-frequency noise generated by environmental electromagnetic fields and electromyography. Finally, the low-frequency trend term of the signal is estimated using median filtering to estimate the baseline. The baseline drift caused by respiration and body movement is eliminated by subtracting the baseline estimation result from the low-pass filtered pulse wave signal. S22. Perform periodic segmentation on the preprocessed pulse wave signal, extract the pulse wave interval sequence between adjacent pulse wave periods, evaluate the effectiveness of the pulse wave period, and remove abnormal periods or pulse wave periods with poor quality. S23. Construct a pulse wave interval sequence based on the retained effective pulse wave period, and extract the time-domain and frequency-domain features of pulse wave variability based on the pulse wave interval sequence; S24. Based on the retained effective pulse wave cycle, analyze the morphological structure of a single pulse wave cycle and extract the morphological characteristic parameters of the pulse wave. The pulse wave features extracted in steps S23 and S24 constitute a pulse wave feature set. .

4. The sleep quality assessment method based on multimodal features in a waking state according to claim 3, characterized in that, In step S22, based on the preprocessed pulse wave signal, the position of the main peak of the pulse wave is first detected, and the corresponding pulse wave foot point is located within a preset time window before each main peak. The foot point serves as the boundary point of the pulse wave period. After periodically segmenting the pulse wave signal, a pulse wave interval sequence is constructed. Then, the segmented pulse wave signal undergoes morphological consistency screening. By performing correlation analysis between the morphology of a single-cycle pulse wave signal and a reference template, cycles with significant morphological distortion are eliminated. The reference template is obtained through the following steps: In the set of effective pulse cycles that meet the physiological rationality constraints, each pulse cycle is resampled to a uniform length and its amplitude is standardized. Then, the normalized waveforms of each cycle are statistically analyzed point by point, and the median is taken to form a representative cycle waveform. Using this representative cycle waveform as a reference template, the correlation coefficient between any cycle to be evaluated and the reference template is calculated. Cycles with a correlation coefficient lower than a preset threshold are judged as morphological abnormalities and are removed. In step S23, based on the removal of distorted cycles, the remaining pulse wave interval sequence is subjected to cubic spline interpolation resampling to compensate for the discontinuity caused by the removal of abnormal cycles, and a pulse wave interval variation sequence with equal time intervals is obtained. Based on this sequence, indicators reflecting the characteristics of autonomic nervous activity are calculated, including time domain features and frequency domain features. In step S24, based on the single-cycle waveform that has passed the effectiveness assessment, the main peak, diphtheria notch, and diphtheria peak are identified using the first and second derivatives. By calculating the time-domain coordinates and amplitudes of the above feature points, morphological parameters reflecting the cardiovascular state are extracted, including rise / fall characteristics and amplitude characteristics.

5. The sleep quality assessment method based on multimodal features in a waking state according to claim 3, characterized in that, Step S3 specifically includes: S31. Preprocess the acquired facial thermal imaging images and corresponding temperature matrices, including removing invalid data frames and aligning timestamps. Invalid data frames include key point occlusion, missing temperature matrix, and missing facial images. S32. Use a key point detection model to identify facial key points and obtain the coordinates of key points on the forehead, around the eyes, and the tip of the nose; based on the geometric relationship of the key points and adaptive constraints on face size, determine the coordinate range of the region of interest (ROI). S33. Based on the determined regions of interest (ROIs), extract the temperature values ​​of the pixels within the corresponding temperature matrix, and construct the temperature time series of each region of interest according to the acquisition time order; extract facial temperature features that characterize the temperature level and temporal fluctuation characteristics of local facial regions. The facial temperature features include temperature level statistical features, temperature dispersion features, temporal dynamic features, and regional symmetry and regional difference features. The extracted facial temperature features are used to construct a facial temperature feature set. .

6. The sleep quality assessment method based on multimodal features in a waking state according to claim 5, characterized in that, In step S32, a key point detection model is used to infer the key points of the forehead in each frame of facial thermal imaging image. Key points around the right eye Key points around the left eye and key points of the tip of the nose Pixel coordinates; The interocular distance is calculated as a facial scale parameter based on the coordinates of key points around the left and right eyes. The calculation formula is as follows: ； Centered on any keypoint P(x,y), based on the face scale parameter d eye The width and height of the ROI are determined proportionally, and a vertical offset is introduced to adapt to different facial areas; the formula for calculating the ROI boundary coordinates is: ； ； ； ； in, This is the ROI width scaling factor. This is the ROI height ratio coefficient. This is the vertical offset scaling factor.

7. The sleep quality assessment method based on multimodal features in a waking state according to claim 5, characterized in that, Step S4 is as follows: Based on the raw PVT behavioral data, response behavior data were obtained, and behavioral feature parameters characterizing the subjects' alertness level and attention maintenance ability were extracted. The extracted behavioral feature parameters constituted the PVT behavioral feature set. .

8. The sleep quality assessment method based on multimodal features in a waking state according to claim 7, characterized in that, Step S5 is as follows: pulse wave feature set Facial temperature feature set and PVT behavioral feature set The dataset is constructed by fusing the scores of a sleep quality scale with preset thresholds as sample labels. For subjects with poor sleep quality, the sample composition is as follows: Input features Sample Labels ; For subjects with good sleep quality, the sample composition is as follows: Input features Sample Labels .

9. The sleep quality assessment method based on multimodal features in a waking state according to claim 8, characterized in that, Step S6 specifically includes: S61. Use the dataset constructed in step S5 as the sample training set. During the model training process, use K-fold cross-validation to divide the training sample set into multiple subsets. During the training process, select a subset as the validation set in turn, and use the remaining subset as the training set for the training and validation of the classifier. S62. Based on the verification partitioning results of step S61, the classifier is trained using a random forest binary classification model and verified on the validation set, thereby learning the mapping relationship between input features and sample labels. S63. Based on the training and validation results, comprehensively evaluate the classifier performance and determine the final sleep quality classification model.

10. The sleep quality assessment method based on multimodal features in a waking state according to claim 1, characterized in that, Step S7 specifically includes: S71. For test subjects who are unaware of their own sleep quality, collect the user's pulse physiological behavior data according to step S1. S72. Process the user's physiological behavior data according to steps S2 to S5 to obtain the user's input features. ; S73. Input the user's input features into the sleep quality classification model to assess the user's nighttime sleep quality while the user is awake during the day.

Images