Sleep awakening detection and evaluation method and device based on radar and PPG signals

Abstract

The invention provides a sleep awakening detection and evaluation method and device based on radar and PPG signals. The method is applied to the technical field of radar and biological signal data processing. The method comprises the following steps: acquiring a radar echo signal and a photoelectric volume pulse wave signal acquired during a sleep period of a target object; performing multi-scale feature extraction on the radar spectrogram by using a first multi-scale feature extraction network, performing multi-scale feature extraction on the photoelectric volume pulse wave time-frequency graph by using a second multi-scale feature extraction network, and fusing a radar multi-scale feature graph and a pulse multi-scale feature graph by using a fusion layer; performing regression classification on the multi-scale fusion feature map by using a fragment proposal network task head to obtain candidate feature picture segments; and performing sleep awakening event prediction on the candidate feature picture segment by using the interested segment head network to obtain a target feature picture segment. According to the invention, the accuracy of sleep awakening event detection is improved.

Description

Technical Field

[0001] This invention relates to the field of radar and biosignal data processing technology, specifically to a method and device for sleep-wake detection and assessment based on radar and PPG signals. Background Technology

[0002] Population aging has become a global challenge. In the fields of healthcare and elderly care, sleep health is crucial, with sleep arousal, as a key indicator for assessing sleep quality and diagnosing diseases, receiving widespread attention. According to the American Academy of Sleep Medicine, sleep is divided into three stages: W (wakeful), R (rapid eye movement) sleep, and N1-N3 (non-rapid eye movement) sleep. Sleep arousal events refer to brief awakenings during sleep, typically lasting more than 3 seconds. These events cause sleep fragmentation and are typical symptoms of insomnia, sleep apnea syndrome, and other disorders. Frequent awakenings reduce sleep quality, increase daytime sleepiness, cognitive decline, and the risk of cardiovascular disease.

[0003] Current sleep-wake detection technologies primarily focus on two approaches: those based on the time-frequency information of electroencephalogram (EEG) signals and those based on cardiovascular and autonomic nervous system signals. However, both approaches have limitations in practical applications. The EEG-based approach suffers from inconvenient device wearing and susceptibility to signal interference, leading to unstable data acquisition and difficulty in consistently obtaining accurate and effective EEG signals, thus severely impacting the accuracy of sleep-wake detection. Similarly, the cardiovascular and autonomic nervous system-based approach is susceptible to environmental interference, and individual differences can also affect the results. These factors make it difficult for this method to accurately reflect sleep-wake states, further reducing the accuracy of sleep-wake detection. Summary of the Invention

[0004] In view of this, the present invention provides a sleep-wake detection method based on radar and PPG signals, comprising: acquiring radar echo signals and photoplethysmography (PPG) signals collected during the sleep of a target object; generating a radar spectrum based on the radar echo signals, converting the PPG signals using short-time Fourier transform to obtain a PPG time-frequency map; extracting multi-scale features from the radar spectrum using a first multi-scale feature extraction network to obtain a radar multi-scale feature map, extracting multi-scale features from the PPG time-frequency map using a second multi-scale feature extraction network to obtain a pulse multi-scale feature map, and fusing the radar multi-scale feature map and the pulse multi-scale feature map using a fusion layer to obtain a multi-scale fused feature map; performing regression classification on the multi-scale fused feature map using a fragment proposal network task head to obtain candidate feature image segments; and using a fragment of interest (FOO) head network to predict sleep-wake events from the candidate feature image segments to obtain target feature image segments containing sleep-wake events.

[0005] Optionally, the fragment proposal network task head includes a fully connected layer group, a regression task head, a classification task head, and a horizontal anchor box generator. The fragment proposal network task head is used to predict sleep-wake events on the multi-scale fused feature map to obtain candidate feature image segments. This includes: using the fully connected layer group to highly compress the multi-scale fused feature map to obtain a fixed-height multi-scale fused feature map; using the regression task head to predict anchor box offsets at each time step of the fixed-height multi-scale fused feature map to obtain the predicted values ​​of the first anchor box width offset and the first anchor box center point x-coordinate offset; using the classification task head to determine whether there are sleep-wake events in the anchor boxes of the fixed-height multi-scale fused feature map at each time step, thereby determining the anchor box category; using the horizontal anchor box generator to generate multi-scale anchor boxes in the width direction of the fixed-height multi-scale fused feature map; and using the horizontal anchor box generator to process the multi-scale anchor boxes based on the predicted value of the first anchor box width offset, the first anchor box center point x-coordinate offset, and the corresponding anchor box category to obtain candidate feature image segments.

[0006] Optionally, the multi-scale anchor frames are processed based on the predicted value of the width offset of the first anchor frame, the horizontal coordinate offset of the center point of the first anchor frame, and the corresponding anchor frame category to obtain candidate feature image segments. This includes: correcting the position and size of the multi-scale anchor frames based on the predicted value of the width offset of the first anchor frame and the horizontal coordinate offset of the center point of the first anchor frame to obtain corrected multi-scale anchor frames; and filtering the corrected multi-scale anchor frames based on the anchor frame category to obtain candidate feature image segments.

[0007] Optionally, filtering the modified multi-scale anchor boxes based on anchor box categories to obtain candidate feature image segments includes: determining the confidence level of the sleep-wake event corresponding to the anchor box in each scale layer based on the anchor box category; selecting the anchor boxes with the highest confidence level from the anchor boxes in each scale layer based on a preset number; removing the anchor boxes in each scale layer after filtering based on a preset width threshold, and performing non-maximum suppression on the remaining anchor boxes in each scale layer to obtain the effective anchor boxes in each scale layer; selecting the effective anchor boxes with the highest confidence level from the effective anchor boxes in each scale layer based on a preset number, and determining the feature image segments corresponding to the retained effective anchor boxes as candidate feature image segments.

[0008] Optionally, the segment of interest (SIO) head network includes a wakefulness filter, a region of interest (ROI) alignment layer, a multilayer perceptron, and a network prediction head. The SIO head network is used to predict sleep-wake events from candidate feature image segments to obtain target feature image segments containing sleep-wake events. This includes: filtering the candidate feature image segments for wakefulness using the wakefulness filter to obtain sleep-wake feature image segments; using the ROI alignment layer to truncate the multi-scale fused feature map based on the sleep-wake feature image segments to obtain fixed-size feature image segments; extracting key features from the fixed-size feature image segments using the multilayer perceptron to obtain key feature image segments; and using the network prediction head to predict sleep-wake events from the key feature image segments to obtain target feature image segments containing sleep-wake events.

[0009] Optionally, the network prediction head is used to predict sleep-wake events in key feature image segments to obtain target feature image segments containing sleep-wake events. This includes: using the network prediction head to predict anchor frame offsets at each time step of the key feature image segment, obtaining predicted values ​​for the second anchor frame width offset and the second anchor frame center point x-coordinate offset; using the network prediction head to predict the confidence of sleep-wake events in the key feature image segment, obtaining the confidence of the key feature image segment; using the network prediction head, based on the predicted values ​​for the second anchor frame width offset and the second anchor frame center point x-coordinate offset, correcting the position and size of anchor frames in candidate feature image segments to obtain corrected feature image segments; and filtering the corrected feature image segments based on the confidence of the key feature image segments to obtain target feature image segments containing sleep-wake events.

[0010] Optionally, the awake period filter is used to filter the candidate feature image segments to obtain the sleep period feature image segments, including: using the awake period filter to calculate the temporal overlap between the candidate feature image segments and the awake period segments; using the awake period filter to remove candidate feature image segments that exceed the preset overlap based on the temporal overlap, thereby obtaining the sleep period feature image segments.

[0011] This invention provides a sleep-wake detection method based on radar and PPG signals, further comprising: acquiring radar echo signals and photoplethysmography (PPG) signals for training; processing the radar echo signals and PPG signals for training respectively to obtain training radar spectra and training PPG time-frequency maps; slicing the training radar spectra and training PPG time-frequency maps respectively to obtain radar spectra slice datasets and PPG time-frequency map slice datasets; using a first multi-scale feature extraction network to extract features from the radar spectra slice dataset to obtain a training radar multi-scale feature map; using a second multi-scale feature extraction network to extract multi-scale features from the PPG time-frequency map slice dataset to obtain a training pulse multi-scale feature map; and then using a fusion method... The training layer fuses the training radar multi-scale feature map and the training pulse multi-scale feature map to obtain a training multi-scale fused feature map. The fragment proposal network task head is used to perform regression classification on the training multi-scale fused feature map to obtain training candidate feature image segments, and a first loss function is calculated for the fragment proposal network task head's regression classification. The segment of interest head network is used to predict sleep-wake events using the training candidate feature image segments, and a second loss function is calculated for the segment of interest head network's sleep-wake event prediction. The network parameters of the fragment proposal network task head are updated based on the first loss function, and the network parameters of the segment of interest head network are updated based on the second loss function, thus completing the training of the fragment proposal network task head and the segment of interest head network.

[0012] A second aspect of the present invention provides a sleep-wake assessment method based on radar and PPG signals, comprising: performing sleep-wake event statistics on target feature image segments obtained by the aforementioned sleep-wake detection method based on radar and PPG signals to obtain the number of sleep-wake events; calculating a sleep-wake index based on the sleep duration and the number of sleep-wake events of the target object; and assessing the sleep quality of the target object based on the sleep-wake index.

[0013] A third aspect of the present invention provides a sleep-wake detection device based on radar and PPG signals. The device includes a processor and a memory connected to the processor. The memory stores instructions that can be executed by the processor, and the instructions are executed by the processor to cause the processor to perform the aforementioned sleep-wake detection method based on radar and PPG signals.

[0014] This invention processes the acquired radar echo signal and photoplethysmography (PPG) signal separately to obtain radar spectra and PPG time-frequency maps. These two feature maps are then input into a deep neural detection network. A first multi-scale feature extraction network extracts multi-scale features from the radar spectra to obtain a radar multi-scale feature map. A second multi-scale feature extraction network extracts multi-scale features from the PPG time-frequency map to obtain a pulse multi-scale feature map. A fusion layer then fuses the radar and pulse multi-scale feature maps to obtain a multi-scale fused feature map. By fusing the multi-modal features of the radar spectra and PPG time-frequency maps, this invention effectively compensates for the susceptibility of single signal sources to noise interference in complex environments, which can lead to misjudgments and missed detections. This significantly enhances the stability and reliability of sleep-wake event detection in various application scenarios. Finally, a segment proposal network task head is used to perform regression classification on the multi-scale fused feature map, efficiently and accurately selecting candidate feature image segments with potential value. Finally, the candidate feature image segments are used to predict sleep-wake events using the segment of interest head network, resulting in target feature image segments containing sleep-wake events. This network will then re-evaluate the candidate feature segments, thereby reducing the false detection rate and improving the accuracy of sleep-wake event detection. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a sleep-wake detection method based on radar and PPG signals in an embodiment of the present invention; Figure 2 This is a structural diagram of the deep neural detection network in an embodiment of the present invention; Figure 3 This is a structural diagram of the first multi-scale feature extraction network and the second multi-scale feature extraction network in an embodiment of the present invention; Figure 4 This is a structural diagram of the residual identity block and the residual projection block in an embodiment of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0019] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can also refer to the internal connection of two components; and they can refer to a wireless connection or a wired connection. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0020] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0021] like Figure 1 As shown, this embodiment of the invention provides a sleep-wake detection method based on radar and PPG signals. This method is executed by an electronic device such as a computer or server, and specifically includes: S1, acquire radar echo signals and photoplethysmography (PPG) signals collected during the target object's sleep period.

[0022] The specific acquisition scenarios for radar echo signals and photoplethysmography (PPG) signals in this embodiment are as follows: Millimeter-wave radar equipment is installed at an appropriate location on the roof of the radar monitoring scenario to collect data on the target object throughout the night for sleep-wake detection. During monitoring, the millimeter-wave radar transmits a frequency-modulated continuous wave (FMCW) signal towards the target object. When the transmitted FMCW signal encounters the target object, a portion of the signal is reflected back, forming an echo signal. The received echo signal is mixed with the transmitted signal to obtain a difference frequency signal, which is then high-pass filtered, low-noise amplified, and sampled by an ADC to obtain a digitized radar echo signal.

[0023] While the target subject sleeps, they wear a device on their wrist, such as a watch, bracelet, or ring, that can collect human pulse waves. These devices use built-in optical sensors to detect changes in blood oxygenation and flow rate as blood flows through the arteries in the wrist, which alter the intensity of reflected light. By continuously monitoring these minute changes in light intensity, the optical sensors can identify the generation and propagation of pulse waves, thereby acquiring the target subject's photoplethysmography (PPG) signal. This non-invasive data acquisition method achieves contactless vital sign data collection, minimizing interference with the target subject and contributing to the acquisition of more accurate sleep data, thus improving the quality of sleep monitoring and the comfort of the target subject.

[0024] S2, a radar spectrum is generated based on the radar echo signal, and the photoplethysmography (PPG) signal is converted using short-time Fourier transform to obtain the PPG time-frequency diagram.

[0025] The radar spectrum in this embodiment specifically includes body motion intensity. respiratory intensity and breathing Doppler The radar spectrum generation process involves first performing a Fourier transform on the radar echo signal to obtain a complex-valued time-range image. This image is then processed to obtain a respiratory phase-time map, which can be used to describe human respiration. Next, the original radar echo signal is processed into three physically meaningful spectra: a body motion intensity spectrum. respiratory intensity spectrum Respiratory Doppler spectrum These data, with distance *r* and time *t* as independent variables, reflect the distribution of corresponding physical quantities with respect to distance and time. The physical meanings of the three radar spectra are as follows: [Image of volumetric intensity spectrum] Indicates the intensity of limb movements, turning over, and other bodily movements of the target object; respiratory intensity spectrum. Indicates the intensity of respiratory movements of the target object; respiratory Doppler spectrum This represents the product of the target object's respiratory Doppler frequency and respiratory intensity.

[0026] For photoplethysmography (PPG) signals, considering their subtle changes and complex time-frequency characteristics during wakefulness, a short-time Fourier transform is used to convert them into a PPG time-frequency map. This operation transforms a one-dimensional time-domain signal into an image containing three-dimensional information of time, frequency, and intensity, making it easier for convolutional networks to extract arousal-related feature patterns. (Photoplethysmography pulse wave time-frequency map) This represents the frequency components and intensity distribution of the photoplethysmography (PPG) signal of the target object at different time points. Specifically, the window function type, window length, and step size of the short-time Fourier transform were optimized to suit the characteristics of the PPG signal, ensuring that the PPG time-frequency map and the radar spectrum are aligned in time steps. Specifically, a Hanning window was used, with a window duration of 100 sampling points (corresponding to a duration of 10 seconds), a frame shift of 1 sampling point (corresponding to a duration of 0.1 seconds), and a sampling rate of 10Hz. This specific parameter combination effectively preserves the time-frequency characteristics of micro-awakening events, providing a crucial foundation for the model's high-precision detection.

[0027] S3. The radar spectrum is extracted using the first multi-scale feature extraction network to obtain a radar multi-scale feature map. The photoplethysmography pulse wave time-frequency map is extracted using the second multi-scale feature extraction network to obtain a pulse multi-scale feature map. The radar multi-scale feature map and the pulse multi-scale feature map are then fused using a fusion layer to obtain a multi-scale fused feature map.

[0028] Figure 2 This is a structural diagram of the deep neural network for sleep and wakefulness detection (Radar and PPG for Arousal detection R-CNN, RAPA R-CNN) in this embodiment of the invention, including a backbone network 1, a segment proposal network task head 2, and a segment of interest head network 3. Step S2 specifically involves inputting the radar spectrogram and the photoplethysmography (PPG) time-frequency image in parallel into the deep neural network. The first multi-scale feature extraction network 11 of the backbone network 1 performs multi-scale local feature extraction on the radar spectrogram, and the second multi-scale feature extraction network 12 performs multi-scale local feature extraction on the PPG time-frequency image. Finally, the fusion layer 13 merges the radar multi-scale feature image and the PPG multi-scale feature image into a single feature image in the channel dimension according to the scale hierarchy of the feature images, resulting in a multi-scale fused feature image.

[0029] like Figure 3As shown, both the first and second multi-scale feature extraction networks adopt a structure combining ResNet-18 (Residual Neural Network–18, 18-layer residual neural network) and Feature Pyramid Network (FPN). The first multi-scale feature extraction network is used for multi-scale feature extraction of three-channel radar spectra, and its input layer has three channels. The second multi-scale feature extraction network is used for multi-scale feature extraction of single-channel time-frequency maps, and its input layer has a single channel.

[0030] The 18-layer residual neural network includes a first 2D convolutional layer, a first max-pooling layer, a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer. The first 2D convolutional layer has a 7×7 kernel with a stride of s²=2; the max-pooling layer has a 3×3 kernel with a stride of s²=2; the first convolutional layer includes two residual identity blocks; the second, third, and fourth convolutional layers each include a residual identity block and a residual projection block. The feature pyramid network includes four second 2D convolutional layers, three upsampling layers, four third 2D convolutional layers, and a second max-pooling layer. The four second 2D convolutional layers all have a 1×1 kernel with a stride of s¹=1; the four third 2D convolutional layers all have a 3×3 kernel with a stride of s¹=1; and the second max-pooling layer all have a 1×1 kernel with a stride of s²=2.

[0031] like Figure 4 As shown, the residual identity block includes two third 2d convolutional layers, two batch normalization layers, and a modified linear unit. The residual projection block includes a fourth 2d convolutional layer (all convolutional kernels are 3×3, stride is s2=2), two batch normalization layers, a modified linear unit, a third 2d convolutional layer, and a fifth 2d convolutional layer (all convolutional kernels are 1×1, stride is s2=2).

[0032] S4 utilizes the fragment proposal network task head to perform regression classification on the multi-scale fused feature map to obtain candidate feature image segments.

[0033] like Figure 2As shown, the Segment Proposals Network Head (SPN Head) in the Deep Neural Detection Network is designed based on the Region Proposal Network Head (RPN Head). By regressing and classifying the multi-scale fused feature map, it determines the specific location of candidate feature image segments in the multi-scale fused feature map and judges whether these segments contain sleep-wake events, thereby efficiently filtering out candidate feature image segments with potential value from the multi-scale fused feature map.

[0034] S5. The segment of interest head network is used to predict sleep-wake events from candidate feature image segments, resulting in target feature image segments containing sleep-wake events.

[0035] like Figure 2 As shown, the Segments of Interest (SoI Heads) network 3 in the deep neural detection network is designed based on the Regions of Interest (RoI Heads) network. It predicts sleep-wake events by fusing multi-scale feature maps, specifically by accurately classifying candidate feature segments into sensory region segments and fine-tuning their bounding boxes. Finally, it outputs the confidence level of the sleep-wake event and its precise start and end times. By re-predicting through the Regions of Interest head network, the false positive rate is reduced, improving the accuracy of sleep-wake event detection.

[0036] This embodiment processes radar echo signals and photoplethysmography (PPG) signals separately to obtain radar spectra and PPG time-frequency maps. These two feature maps are then input into a deep neural detection network. A first multi-scale feature extraction network extracts multi-scale features from the radar spectra, resulting in a radar multi-scale feature map. A second multi-scale feature extraction network extracts multi-scale features from the PPG time-frequency map, resulting in a pulse multi-scale feature map. A fusion layer then fuses the radar and pulse multi-scale feature maps to obtain a multi-scale fused feature map. By fusing the multi-modal features of the radar spectra and PPG time-frequency maps, the problem of single signal sources being susceptible to noise interference in complex environments, leading to misjudgments and missed detections, is effectively mitigated. This significantly enhances the stability and reliability of sleep-wake event detection in various application scenarios. Finally, a segment proposal network task head is used to perform regression classification on the multi-scale fused feature map, efficiently and accurately selecting candidate feature image segments with potential value. Finally, the candidate feature image segments are used to predict sleep-wake events using the segment of interest head network, resulting in target feature image segments containing sleep-wake events. This network will then re-evaluate the candidate feature segments, thereby reducing the false detection rate and improving the accuracy of sleep-wake event detection.

[0037] like Figure 2 As shown, the fragment proposal network task head 2 includes a fully connected layer group 21, a regression task head 22, a classification task head 23, and a horizontal anchor box generator 24.

[0038] Furthermore, step S4, which utilizes the fragment proposal network task head to perform regression classification on the multi-scale fused feature map to obtain candidate feature image segments, mainly includes: S41 utilizes fully connected layers to highly compress the multi-scale fusion feature map, resulting in a multi-scale fusion feature map with a fixed height.

[0039] In this embodiment, the fully connected layer group 21 includes five fully connected layers with different input dimensions and the same output dimension. Each of the five fully connected layers compresses the height (H) of the feature maps at different scales to 1. The specific operation is as follows: first, the height (H) and channel dimension (C) of the five scale feature maps are merged. Then, through the mapping method of the fully connected layers, the dimension (C×H) of the merged feature maps is uniformly mapped to a fixed dimension (C×4). Next, the fixed-dimensional feature map is split into feature maps with height H=1 and channel dimension C×4. Finally, the format of the fixed-height multi-scale fusion feature map is [B, 4C, 1, W]. By using the fully connected layer group to highly compress the multi-scale fusion feature map, the two-dimensional feature map is compressed into a one-dimensional time series before processing, significantly reducing model complexity and computational resource consumption. This design achieves feature fusion in the vertical dimension while maintaining the spatial structure in the horizontal dimension, allowing the fragment proposal network task head 2 to focus on horizontal localization and scale prediction.

[0040] S42, using the regression task head, anchor frame offset prediction is performed at each time step of the multi-scale fusion feature map at a fixed height, to obtain the predicted value of the first anchor frame width offset and the predicted value of the first anchor frame center point x-coordinate offset.

[0041] The feature maps of five different feature scales with a fixed height output by the fragment proposal network task head 2 will be input into the regression task head 22 of this embodiment. The regression task head 22 will generate a set of anchor frame width offset prediction values ​​and anchor frame center point x-coordinate offset prediction values ​​at each time step of the feature map, so as to adjust the actual anchor frame and improve the detection of sleep wakefulness.

[0042] S43 uses the classification task head to determine whether there are sleep / wake events in the anchor boxes of the fixed-height multi-scale fusion feature map at each time step, in order to determine the anchor box category.

[0043] In this embodiment, the classification task head 23 determines whether there are sleep-wake events in the anchor boxes of the fixed-height multi-scale fusion feature map at each time step, thus obtaining the classification of each anchor box at each time step. This provides a foundation for accurate identification of sleep-wake events. By clearly defining the category of each anchor box, regions containing key sleep-wake information can be effectively distinguished, thereby improving the overall detection accuracy.

[0044] S44 uses a horizontal anchor box generator to generate multi-scale anchor boxes in the width direction of a multi-scale fused feature map at a fixed height.

[0045] The horizontal anchor frame generator 24 in this embodiment has two advantages: First, in terms of dynamic width adaptation, the horizontal anchor frame generator 24 can pre-set a series of anchor frame widths of different sizes (scales) according to the different levels of the feature map, thereby flexibly adjusting the anchor frame width. This allows it to accurately capture sleep-wake events of varying durations from 3 seconds to 30 seconds. Different durations of sleep-wake events are represented differently on the feature map. By dynamically adjusting the anchor frame width, the horizontal anchor frame generator 24 improves the detection capability of sleep-wake events of different durations, ensuring that all types of sleep-wake events can be effectively identified. Second, in terms of reducing computational load, the horizontal anchor frame generator 24 only generates displacement coordinates in the horizontal direction, and the displacement in the vertical direction is always zero. This one-dimensional displacement design avoids unnecessary calculations in the vertical direction. When generating multi-scale anchor frames in the width direction of a multi-scale fused feature map at a fixed height, the total number of anchor frames is greatly reduced, and anchor frame generation can be completed in a shorter time.

[0046] S45, using a horizontal anchor frame generator, multi-scale anchor frames are processed based on the predicted value of the width offset of the first anchor frame, the horizontal coordinate offset of the center point of the first anchor frame, and the corresponding anchor frame category to obtain candidate feature image segments.

[0047] The core of this step lies in finely adjusting and filtering the multi-scale anchor frames generated by the horizontal anchor frame generator 24 to obtain candidate feature image segments.

[0048] This embodiment utilizes fully connected layers to highly compress multi-scale fused feature maps, transforming two-dimensional feature maps into one-dimensional time series. This reduces model complexity and computational resource consumption, achieving vertical feature fusion while preserving horizontal spatial structure, allowing the model to focus on horizontal localization and scale prediction. Next, a regression task head predicts anchor box offsets at each time step of the multi-scale fused feature map at a fixed height, adjusting the actual anchor boxes and improving the accuracy of sleep-wake detection. Then, a classification task head determines whether anchor boxes contain sleep-wake events, identifying the anchor box category and providing a foundation for accurate identification of sleep-wake events, distinguishing key information regions, and improving overall detection accuracy. A horizontal anchor box generator generates multi-scale anchor boxes in the width direction; its dynamic width adaptation accurately captures sleep-wake events of different durations, and the one-dimensional displacement design reduces computation and shortens anchor box generation time. Finally, based on the predicted values ​​and anchor box categories, the multi-scale anchor boxes are processed to achieve fine-tuning and filtering, obtaining candidate feature image segments. The segment proposal network task head further improves detection accuracy and precision.

[0049] In some optional embodiments of this example, step S45, which processes the multi-scale anchor frames based on the predicted value of the first anchor frame width offset, the horizontal coordinate offset of the first anchor frame center point, and the corresponding anchor frame category to obtain candidate feature image segments, mainly includes: S451, Based on the predicted value of the width offset of the first anchor frame and the horizontal coordinate offset of the center point of the first anchor frame, the position and size of the multi-scale anchor frame are corrected to obtain the corrected multi-scale anchor frame.

[0050] Based on the predicted value of the width offset of the first anchor frame and the horizontal coordinate offset of the center point of the first anchor frame, the original position and size of the multi-scale anchor frame are adjusted and modified to obtain a corrected multi-scale anchor frame whose position and size are more in line with the actual situation.

[0051] S452, based on the anchor box category, the modified multi-scale anchor boxes are filtered to obtain candidate feature image segments.

[0052] Based on the anchor box category information, the modified multi-scale anchor boxes are filtered to remove those that do not meet the requirements or are unlikely to contain the target, and finally valuable candidate feature image segments are obtained.

[0053] This embodiment corrects the position and size of multi-scale anchor frames based on the predicted value of the first anchor frame width offset and the x-coordinate offset of the first anchor frame center point, enabling the anchor frames to more accurately match the actual target. Next, filtering is performed based on anchor frame category, removing a large number of invalid or irrelevant anchor frames and reducing the amount of data processed subsequently. By processing multi-scale anchor frames, the likelihood of candidate feature image segments containing sleep-wake events is increased, improving the efficiency and accuracy of the entire sleep-wake detection process.

[0054] In some optional implementations of this embodiment, the process of filtering the modified multi-scale anchor boxes based on the anchor box category to obtain candidate feature image segments in step S452 mainly includes: S4521, determine the confidence level of the sleep-wake event corresponding to the anchor frame in each scale layer based on the anchor frame category.

[0055] In each layer of feature map, the classification of each corrected anchor box needs to be confirmed with confidence, which represents the probability that the anchor box contains a sleep-wake event.

[0056] S4522, based on a preset number of anchor frames in each scale layer, selects the anchor frames with the highest confidence level.

[0057] The modified anchor boxes in each feature map are sorted from high to low confidence, and only the top N anchor boxes are retained. This allows us to prioritize the anchor boxes that are most likely to contain sleep-wake events and initially screen out high-quality candidate anchor boxes.

[0058] S4523, based on a preset width threshold, remove the anchor frames of each scale layer after filtering, and perform non-maximum suppression operation on the remaining anchor frames of each scale layer to obtain the effective anchor frames of each scale layer.

[0059] Because anchor boxes with excessively small widths may not fully cover the features of sleep-wake events, or may only capture some insignificant small areas, they do not significantly contribute to accurate target identification. Therefore, this embodiment removes anchor boxes with widths smaller than a preset width threshold, further eliminating those anchor boxes that are unlikely to contain valid sleep-wake event features, thus improving the quality of candidate anchor boxes. Simultaneously, in each feature map layer, multiple anchor boxes may enclose the same sleep-wake event, resulting in overlap between these anchor boxes. Non-maximum suppression removes other anchor boxes that have a high degree of overlap with the anchor box with the highest confidence level, retaining only the anchor box with the highest confidence level that represents the sleep-wake event. This avoids repeated detection of the same target, reduces redundant information, and makes candidate anchor boxes more concise and effective.

[0060] S4524: Based on a preset number of valid anchor boxes at each scale layer, select the top-ranked valid anchor boxes by confidence, and determine the feature image segments corresponding to the retained valid anchor boxes as candidate feature image segments.

[0061] After the previous steps of filtering, the remaining valid anchor boxes in all layer feature maps will be obtained. In this embodiment, these anchor boxes will be ranked again according to their confidence level, and then the top N anchor boxes will be selected as the final candidate fragments to ensure that the final candidate fragments are most likely to contain sleep-wake events.

[0062] This embodiment first uses confidence ranking to prioritize and retain the most promising anchor boxes, improving the quality and accuracy of candidate fragments. Next, it removes anchor boxes with excessively small widths, avoiding interference from invalid information and allowing subsequent processing to focus more on valuable regions. Furthermore, it reduces redundant and duplicate detections through non-maximum suppression, lowering computational costs. Finally, it performs confidence ranking again and selects the top few anchor boxes, further ensuring the high quality of the final candidate fragments.

[0063] like Figure 2 As shown, the segment of interest head network 3 includes a wake proposals filter 31, a region of interest alignment layer 32, a two-layer perceptron (Two MLP Head) 33, and a network prediction head (Radar and PPG for Arousal detection R-CNN Predictor, RAPA R-CNN Predictor) 34.

[0064] Furthermore, step S5, which uses the segment of interest head network to predict sleep-wake events from candidate feature image segments to obtain target feature image segments containing sleep-wake events, mainly includes: S51, use the wakefulness filter to filter the candidate feature image segments to obtain the sleep feature image segments.

[0065] Since awakening events specifically refer to physiological phenomena occurring after falling asleep, the detection task naturally focuses on the sleep period (including NREM and REM sleep periods), without needing to detect the wakeful period. During the wakeful period, the target subject is fully awake and there are no micro-awakening phenomena typical of sleep; therefore, such detection at this stage is neither necessary nor targeted. Thus, this embodiment utilizes the wakeful period filter 31, based on sleep stage labels, to filter out candidate feature segments from the wakeful period, ultimately leaving only sleep period feature image segments that may contain sleep-wake events, avoiding invalid calculations during the wakeful period when detection is meaningless.

[0066] S52 utilizes the region of interest alignment layer to extract the segment of interest from the multi-scale fused feature map based on the sleep period feature image segment, resulting in a fixed-size feature image segment.

[0067] Using the region of interest alignment layer 32, based on the event coordinate information carried by the sleep feature image segment after being filtered by the wakefulness filter 31, the multi-scale fusion feature map output by the fusion layer 13 is accurately located and cropped to generate a feature image segment of a fixed size (e.g., (7×7)), providing standardized feature input for subsequent deep analysis and task execution.

[0068] S53 uses a multilayer perceptron to extract key features from a fixed-size feature image segment, thus obtaining a key feature image segment.

[0069] In this embodiment, a multilayer perceptron 33 is used to perform in-depth analysis on fixed-size feature image segments, extract key features closely related to sleep-wake events, remove redundant information, and thus obtain key feature image segments containing key information, laying the foundation for subsequent accurate prediction.

[0070] S54. The network prediction head is used to predict sleep-wake events on key feature image segments to obtain target feature image segments containing sleep-wake events.

[0071] This embodiment utilizes a network prediction head 34 to determine and predict the presence of sleep-wake events based on feature information in key feature image segments. Through analysis of key features, prediction results are output, and target feature image segments that indeed contain sleep-wake events are selected.

[0072] The wakefulness filter in the region of interest (ROI) head network of this embodiment ensures that limited computational resources and attention are focused on the true sleep phase (NREM / REM), effectively avoiding meaningless detections in stages where sleep-wake events are impossible, significantly reducing the false detection rate, and improving overall inference efficiency. Next, a region of interest alignment layer is used to align the size of the sleep phase feature image segments, facilitating subsequent stable processing of the feature segments. Then, a multilayer perceptron is used to extract key features, highlighting information related to sleep-wake events and reducing interference. Finally, the network prediction head is used to make accurate predictions based on the key features, ultimately obtaining the target feature image segment containing sleep-wake events. Further refined regression prediction of candidate feature image segments through the RIO head network improves the efficiency and accuracy of sleep-wake event detection.

[0073] In some optional embodiments of this example, step S54, which uses a network prediction head to predict sleep-wake events in key feature image segments to obtain target feature image segments containing sleep-wake events, mainly includes: S541, the network prediction head is used to predict the anchor frame offset at each time step of the key feature image segment, so as to obtain the predicted value of the second anchor frame width offset and the predicted value of the second anchor frame center point x-coordinate offset.

[0074] This embodiment utilizes a network prediction head to analyze each time step in the key feature image segment. By processing the feature information of these time steps, it predicts the offset of the anchor frame in the width direction and the offset of the center point's horizontal coordinate, obtaining corresponding predicted values. These values ​​can provide a basis for subsequent adjustments to the position and size of the anchor frame.

[0075] S542 uses a network prediction head to predict the confidence of sleep-wake events in key feature image segments, thus obtaining the confidence of the key feature image segments.

[0076] This embodiment utilizes a network prediction head to comprehensively evaluate key feature image segments and determine the probability that the image segment contains a sleep-wake event. The confidence score reflects the degree of association between the key feature image segment and the sleep-wake event.

[0077] S543, using the network prediction head, corrects the position and size of the anchor frame in the candidate feature image segment based on the predicted value of the second anchor frame width offset and the predicted value of the second anchor frame center point x-coordinate offset, to obtain the corrected feature image segment.

[0078] Based on the predicted values ​​of the second anchor frame width offset and the second anchor frame center point x-coordinate offset obtained in the previous steps, the network prediction head will adjust the anchor frames in the candidate feature image segments. By changing the width and center point position of the anchor frames, the anchor frames can more accurately frame possible sleep and wakefulness events, thereby obtaining the corrected feature image segments.

[0079] S544, the modified feature image segments are filtered based on the confidence of the key feature image segments to obtain the target feature image segments containing sleep-wake events.

[0080] The modified feature image segments are filtered based on the confidence level of the key feature image segments. Candidate feature segments with excessively small widths are removed, and non-maximum suppression is applied to these segments (without further feature layer separation). Finally, all remaining candidate feature segments are filtered by confidence level to obtain the target feature image segments containing sleep-wake events. The specific correction steps are the same as those in step S452 and will not be repeated here.

[0081] This embodiment predicts anchor box offsets at each time step of key feature image segments using a network prediction head, enabling more precise anchor box localization to potential sleep-wake event regions. Confidence prediction is then performed on the key feature image segments, providing a quantitative basis for subsequent filtering. Next, the anchor box position and size are adjusted based on the offset prediction values, resulting in a higher match between the anchor boxes and actual sleep-wake events. Finally, filtering is performed based on the confidence scores to remove low-probability image segments, reducing false positives. These steps effectively improve the detection accuracy of sleep-wake events, concentrating computational resources on more valuable targets and improving the efficiency of the entire detection process.

[0082] In some optional embodiments of this example, the process of using a wakefulness filter to filter candidate feature image segments to obtain sleep feature image segments in step S51 mainly includes: S511, using the lucidity filter to calculate the temporal overlap between candidate feature image segments and lucidity segments.

[0083] By using a lucidity filter to analyze each candidate feature image segment and lucidity segment over time, the overlapping portion of the candidate feature image segment and lucidity segment on the time axis can be determined.

[0084] S512, using the wakefulness filter, removes candidate feature image segments that exceed the preset overlap based on the time overlap, and obtains sleep feature image segments.

[0085] The wakefulness filter compares the calculated temporal overlap with a preset overlap threshold (e.g., 0.5). When the overlap between a candidate feature image segment and the wakefulness period exceeds the preset threshold, it means that the candidate feature image segment spends most of its time in the wakefulness period and is likely not to contain sleep-wake events, so it is discarded. Ultimately, only sleep-wake event feature image segments are retained, avoiding invalid calculations during the wakefulness period when there is no detection significance. Specific filtering conditions are as follows: , in, This indicates the temporal overlap between candidate feature image segments and lucidity segments. Indicates the duration of the candidate feature image segment. Indicates the duration of the lucid segment.

[0086] This embodiment uses a wakefulness filter to remove candidate feature image segments with a preset overlap, resulting in sleep-period feature image segments. This accurately excludes candidate feature image segments that highly overlap with wakefulness and are unsuitable for sleep-wake event detection, preventing the model from performing invalid calculations during wakefulness periods when they are not relevant to detection. This not only reduces the possibility of false detections but also allows the model to focus more attention and resources on sleep-period feature image segments that contain sleep-wake events, thereby significantly improving the efficiency and accuracy of sleep-wake event detection.

[0087] In some optional embodiments of this example, the training process of the deep neural network for sleep-wake detection in this embodiment of the invention specifically includes: Step 1: Acquire radar echo signals and photoplethysmography (PPG) signals for training.

[0088] Millimeter-wave radar is used to acquire radar echo signals of any target object during its sleep period, and pulse acquisition equipment is used to acquire photoplethysmography (PPG) signals of any target object during its sleep period.

[0089] Step two involves processing the radar echo signal and photoplethysmography (PPG) signal used for training to obtain the training radar spectrum and the training PPG time-frequency spectrum.

[0090] In this embodiment, the processing of radar echo signals and photoplethysmography (PPG) signals used for training is the same as in step S2, and will not be repeated here.

[0091] Step 3: Slice the training radar spectrum and the training photoplethysmography pulse wave time-frequency map into data slices to obtain the radar spectrum slice dataset and the pulse wave time-frequency map slice dataset.

[0092] The training radar spectrogram and photoplethysmography (PPG) time-frequency map were divided into training, validation, and test sets, respectively. Data slices were then applied to each set to obtain sliced ​​datasets. Specifically, before network training began, to mitigate the issue of excessively large data sets, each data point in both the validation and test sets was sliced ​​using a sliding window. The window length was 18016 sampling points, and the window sliding step was one window length. Before each training iteration, each data point in the training set was randomly cut into two 18016-segment segments, each segment containing at least one sleep-wake event. This random slicing of the training set before each training iteration ensures that the generalization ability of the training set increases with each training iteration, which plays a crucial role in suppressing overfitting and improving network performance.

[0093] Step 4: Use the first multi-scale feature extraction network to extract features from the radar spectrogram slice dataset to obtain the training radar multi-scale feature map. Use the second multi-scale feature extraction network to extract multi-scale features from the pulse wave time-frequency map slice dataset to obtain the training pulse multi-scale feature map. Then, use a fusion layer to fuse the training radar multi-scale feature map and the training pulse multi-scale feature map to obtain the training multi-scale fused feature map.

[0094] The structure and implementation process of the first multi-scale feature extraction network, the second multi-scale feature extraction network, and the fusion layer in this embodiment are the same as in step S3, and will not be repeated here.

[0095] Step 5: Use the fragment proposal network task head to perform regression classification on the trained multi-scale fusion feature map to obtain training candidate feature image segments, and calculate the first loss function for regression classification using the fragment proposal network task head.

[0096] The structure and implementation process of the fragment proposal network task head in this embodiment are consistent with those in step S4, and will not be repeated here. The first loss function calculated in this embodiment includes two parts: an event confidence loss function and a geometric loss function. The event confidence loss function is implemented using Focal Loss. The event confidence loss function has a balancing parameter... and modulation factor The values ​​can be set to 2 and 0.25 respectively. The geometric loss function is implemented using Smooth L1 Loss. Focal Loss is a variant of cross-entropy loss, which enhances the discovery of hard samples and is very helpful for detection tasks with severe imbalance between positive and negative samples, such as sleep / wake detection. Smooth L1 Loss is a combination of L1 and L2 loss functions. This loss function combines the advantages of both L1 and L2 loss functions, solving the problem of gradient explosion in L2 loss function when the error is large and the problem of large gradient values ​​in L1 loss function when the error is small, causing the loss function to fluctuate around the stable value and fail to converge.

[0097] For example, the first loss function L is calculated as follows: , , , , ,

[0098] in, This represents the event confidence loss function. Represents the geometric loss function. This represents the confidence level that the anchor box region predicted by the task head of the fragment proposal network represents a sleep-wake event. This indicates that the balance parameter varies with the positive and negative sample labels. This indicates the probability that the anchor box region predicted by the fragment proposal network task head is an awakening event. Indicates the true label, This represents the difference between the predicted value and the actual value. The threshold represents the geometric loss function.

[0099] Step 6: Use the segment of interest head network to predict sleep-wake events from the trained candidate feature image segments, and calculate the second loss function for sleep-wake event prediction using the segment of interest head network.

[0100] The structure and implementation process of the segment of interest (SIO) head network in this embodiment are consistent with those in step S5, and will not be repeated here. The specific setting method of the second loss function calculated in this embodiment is consistent with that of the first loss function, and will not be repeated here.

[0101] Step 7: Update the network parameters of the segment proposal network task head based on the first loss function, and update the network parameters of the segment of interest head network based on the second loss function, so as to complete the training of the segment proposal network task head and the segment of interest head network.

[0102] During training, the first and second loss functions are continuously minimized to train the task head of the segment proposal network and the head network of segments of interest.

[0103] This embodiment enhances the comprehensiveness and complementarity of data information by combining radar spectrogram slice datasets and pulse wave time-frequency map slice datasets for multi-scale feature extraction and fusion. In training set slice processing, random slicing of the training set before each training round effectively improves the generalization ability of the training set and suppresses overfitting. Regarding the loss function, the event confidence loss function effectively alleviates the extreme positive-negative sample imbalance problem between awakening events and the background, allowing the segment proposal network task head and the segment of interest head network to focus on learning difficult and scarce positive samples; the geometric loss function is used for bounding box regression, solving the gradient explosion problem of the L2 loss function and the convergence difficulty of the L1 loss function, thus improving the overall performance and training stability of the segment proposal network task head and the segment of interest head network.

[0104] This invention also provides a sleep-wake assessment method based on radar and PPG signals. This method is executed by an electronic device such as a computer or server, and specifically includes: Step 1: Based on the target feature image segments obtained by the above-mentioned sleep-wake detection method based on radar and PPG signals, count the number of sleep-wake events.

[0105] Target feature image segments were obtained using a sleep-wake detection method based on radar and PPG signals. These image segments contain key information about the presence of sleep-wake events. This step involves detailed analysis and statistical analysis of these target feature image segments to identify features representing sleep-wake events, and then counting the number of sleep-wake events occurring throughout the entire sleep process.

[0106] Step 2: Calculate the sleep arousal index based on the target subject's sleep duration and the number of sleep arousal events, and evaluate the target subject's sleep quality based on the sleep arousal index.

[0107] For example, the sleep-wake index is calculated using the following method. : , in, Indicates the number of sleep-wake events ( , / Second-rate), Indicates the sleep duration of the target object ( , / h).

[0108] For example, the assessment of sleep quality of a target subject based on the sleep-wake index and the assessment results can be: ≥15, severe; 10≤ <15, moderate; 5≤ <10, mild; <5 is normal.

[0109] This embodiment accurately obtains the number of sleep-wake events by statistically analyzing target feature image segments, ensuring data reliability. A sleep-wake index is calculated based on sleep duration and the number of wakefulness events, and this index is used to assess sleep quality, providing a quantifiable standard for the assessment process and avoiding the arbitrariness of subjective judgment. This assessment method helps users clearly understand their sleep status and provides important evidence for medical personnel to conduct sleep-related diagnoses and treatments, enabling timely detection of potential sleep problems, promoting improved sleep quality, and safeguarding physical health. Through a deep neural network, the entire process from data processing to report generation is automated.

[0110] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0112] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0114] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A radar and PPG signal based sleep-wake detection method, characterized in that, include: Acquire radar echo signals and photoplethysmography (PPG) signals collected during the target's sleep period; A radar spectrum is generated based on the radar echo signal, and the photoplethysmography (PPG) signal is converted using short-time Fourier transform to obtain a PPG time-frequency diagram. The radar spectrum is subjected to multi-scale feature extraction using a first multi-scale feature extraction network to obtain a radar multi-scale feature map. The photoplethysmography pulse wave time-frequency map is subjected to multi-scale feature extraction using a second multi-scale feature extraction network to obtain a pulse multi-scale feature map. The radar multi-scale feature map and the pulse multi-scale feature map are then fused using a fusion layer to obtain a multi-scale fused feature map. The fragment proposal network task head is used to perform regression classification on the multi-scale fused feature map to obtain candidate feature image segments; The candidate feature image segments are used to predict sleep-wake events using a segment of interest head network to obtain target feature image segments containing sleep-wake events.

2. The method of claim 1, wherein, The fragment proposal network task head includes a fully connected layer group, a regression task head, a classification task head, and a horizontal anchor box generator; The method of using the fragment proposal network task head to predict sleep-wake events on the multi-scale fused feature map yields candidate feature image segments, including: The multi-scale fused feature map is highly compressed using the fully connected layer group to obtain a multi-scale fused feature map with a fixed height. Using the regression task head, anchor frame offset prediction is performed at each time step of the multi-scale fusion feature map at a fixed height, to obtain the predicted value of the first anchor frame width offset and the predicted value of the first anchor frame center point x-coordinate offset. The classification task head is used to determine whether there are sleep / wake events in the anchor boxes of the fixed-height multi-scale fusion feature map at each time step, so as to determine the anchor box category. The horizontal anchor frame generator is used to generate multi-scale anchor frames in the width direction of the multi-scale fused feature map at a fixed height. The horizontal anchor frame generator processes the multi-scale anchor frames based on the predicted value of the first anchor frame width offset, the horizontal coordinate offset of the first anchor frame center point, and the corresponding anchor frame category to obtain the candidate feature image segment.

3. The method of claim 2, wherein, Based on the predicted value of the width offset of the first anchor frame, the horizontal coordinate offset of the center point of the first anchor frame, and the corresponding anchor frame category, the multi-scale anchor frame is processed to obtain the candidate feature image segment, including: Based on the predicted value of the width offset of the first anchor frame and the horizontal coordinate offset of the center point of the first anchor frame, the position and size of the multi-scale anchor frame are corrected to obtain the corrected multi-scale anchor frame. The modified multi-scale anchor boxes are filtered based on the anchor box category to obtain the candidate feature image segments.

4. The method of claim 3, wherein, The step of filtering the modified multi-scale anchor boxes based on the anchor box category to obtain the candidate feature image segments includes: The confidence level of the sleep-wake event corresponding to the anchor frame in each scale layer is determined based on the anchor frame category. Based on a preset number, anchor frames with the highest confidence levels are selected from the anchor frames at each scale level; Based on a preset width threshold, the anchor frames of each scale layer after filtering are removed, and non-maximum suppression is performed on the remaining anchor frames of each scale layer to obtain the effective anchor frames of each scale layer. Based on a preset number of valid anchor frames at each scale level, the top-ranked valid anchor frames are selected, and the feature image segments corresponding to the retained valid anchor frames are determined as the candidate feature image segments.

5. The method of claim 1, wherein, The segment of interest head network includes a wakefulness filter, a region of interest alignment layer, a multilayer perceptron, and a network prediction head; The step of using a segment of interest (SIO) head network to predict sleep-wake events from the candidate feature image segments to obtain target feature image segments containing sleep-wake events includes: The candidate feature image segments are filtered for wakefulness using the wakefulness filter to obtain sleep feature image segments. Using the region of interest alignment layer, the multi-scale fused features are cropped based on the sleep period feature image segments to obtain fixed-size feature image segments. The multilayer perceptron is used to extract key features from the fixed-size feature image segment to obtain the key feature image segment. The network prediction head is used to predict sleep-wake events in the key feature image segments to obtain target feature image segments containing sleep-wake events.

6. The method of claim 5, wherein, The step of using the network prediction head to predict sleep-wake events in the key feature image segment to obtain target feature image segments containing sleep-wake events includes: The network prediction head is used to predict the anchor frame offset at each time step of the key feature image segment, so as to obtain the predicted value of the second anchor frame width offset and the predicted value of the second anchor frame center point x-coordinate offset. The confidence score of the key feature image segment is obtained by using the network prediction head to predict the confidence score of the sleep-wake event of the key feature image segment. Using the network prediction head, the position and size of the anchor frame in the candidate feature image segment are corrected based on the predicted value of the second anchor frame width offset and the predicted value of the horizontal coordinate offset of the second anchor frame center point, to obtain the corrected feature image segment; The modified feature image segment is filtered based on the confidence level of the key feature image segment to obtain the target feature image segment containing sleep-wake events.

7. The method of claim 5, wherein, The step of using the wakefulness filter to filter the candidate feature image segments to obtain sleep feature image segments includes: The temporal overlap between the candidate feature image segment and the lucid period segment is calculated using the lucid period filter. Using the wakefulness filter, candidate feature image segments that exceed a preset overlap are removed from the candidate feature image segments based on the time overlap, thus obtaining the sleep period feature image segments.

8. The method according to any one of claims 1 to 7, characterized in that, Also includes: Acquire radar echo signals and photoplethysmography (PPG) signals for training; The radar echo signal and photoplethysmography (PPG) signal used for training are processed separately to obtain the training radar spectrum and the training PPG time-frequency spectrum. The training radar spectrum and the training photoplethysmography pulse wave time-frequency map are sliced ​​to obtain radar spectrum slice dataset and pulse wave time-frequency map slice dataset, respectively. The first multi-scale feature extraction network is used to extract features from the radar spectrogram slice dataset to obtain a training radar multi-scale feature map. The second multi-scale feature extraction network is used to extract multi-scale features from the pulse wave time-frequency map slice dataset to obtain a training pulse multi-scale feature map. The training radar multi-scale feature map and the training pulse multi-scale feature map are then fused using the fusion layer to obtain a training multi-scale fused feature map. The fragment proposal network task head is used to perform regression classification on the trained multi-scale fusion feature map to obtain training candidate feature image segments, and the first loss function of the fragment proposal network task head for regression classification is calculated. The training candidate feature image segments are used to predict sleep-wake events using the segment of interest head network, and a second loss function for sleep-wake event prediction is calculated using the segment of interest head network. The network parameters of the segment proposal network task head are updated based on the first loss function, and the network parameters of the segment of interest head network are updated based on the second loss function, so as to complete the training of the segment proposal network task head and the segment of interest head network.

9. A sleep-wake assessment method based on radar and PPG signals, characterized in that, include: The sleep-wake event count is obtained by performing sleep-wake event statistics on the target feature image segment obtained by the method according to any one of claims 1-8; The sleep arousal index is calculated based on the target subject's sleep duration and the number of sleep arousal events, and the sleep quality of the target subject is evaluated based on the sleep arousal index.

10. A radar and PPG signal based sleep arousal detection device, characterized in that, include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the sleep-wake detection method based on radar and PPG signals as described in any one of claims 1-8.

Images