Automatic sleep staging method based on visual transformer

Abstract

The application discloses an automatic sleep staging method based on a visual Transformer. The original PSG signal is processed through a sliding window to obtain a PSG signal sequence; the PSG signal sequence is processed through data enhancement to obtain an enhanced signal sample; a sleep staging network is constructed through a visual Transformer frame-level encoder, a bidirectional GRU sequence-level encoder and a softmax layer; each group of PSG signal samples is input into the sleep staging network to predict the sleep stage of the PSG signal sample; the network is initialized through cross-modal transfer learning; a loss function is constructed in combination with the real sleep stage of the PSG signal sample; an ADAM optimizer is used for training to obtain an optimized sleep staging network; PSG signals are collected in real time; the PSG signal sample is input into the optimized sleep staging network to predict the sleep stage. The application designs data enhancement according to the noise and artifacts of the PSG signal, improves the robustness of the network to the noise and artifacts of the PSG signal, introduces an encoder based on the visual Transformer to improve the feature representation capability of the network, and reduces the dependence on a large amount of PSG data through transfer learning.

Description

Technical Field

[0001] This invention belongs to the field of sleep quality assessment technology, specifically relating to an automatic sleep staging method based on visual Transformer. Background Technology

[0002] Polysomnography (PSG) is a standard clinical technique for monitoring sleep states. It comprehensively records various physiological indicators during the monitored subject's sleep process, including neural signals such as electroencephalography (EEG), electrooculography (EOG), and electromyography (EMG), as well as respiratory monitoring data such as airflow through the mouth and nose, chest and abdominal pressure, and blood oxygen saturation. It can serve as an effective basis for assessing sleep quality and diagnosing sleep disorder symptoms. However, for a long time, the analysis of sleep physiological signals has relied on sleep experts manually examining polysomnography signals. This method suffers from low efficiency, high labor costs, and the subjective differences in expert knowledge can also lead to errors in the assessment results. Therefore, there is a need to develop robust, high-performance, automated sleep staging tools to assist doctors in sleep staging and improve the efficiency and accuracy of sleep staging.

[0003] Automated sleep staging technology is fundamental to expanding sleep assessment and diagnosis, serving millions of people with sleep disorders and enabling sleep monitoring in the home. While existing automated sleep staging models have achieved impressive performance, surpassing the accuracy of staging by a single human expert, several issues remain to be addressed:

[0004] Visual Transformers can extract effective feature representations, but their performance on PSG signals has not yet been explored; meanwhile, positional information on the time and frequency axes is crucial for PSG signals after Fourier transform. However, recent sleep staging methods only consider positional relationships on the time axis, while visual Transformers can capture positional information on both the time and frequency axes, thus overcoming the shortcomings of existing models.

[0005] Transformer-based deep learning methods require large amounts of training data to outperform CNNs. Current Transformer-based automatic sleep staging models perform well when pre-trained on large-scale PSG datasets, but their accuracy drops significantly on smaller datasets. However, obtaining large-scale, accurately labeled PSG datasets is extremely difficult, and training models from scratch on large PSG datasets requires substantial computational resources.

[0006] Existing automatic sleep staging models are not robust to noise and artifacts. Due to human factors and the acquisition environment, PSG signals inevitably contain noise and artifacts. However, research on data augmentation module design for PSG signals remains very limited. Most works directly adopt augmentation techniques used in image and audio tasks without considering the inherent characteristics of PSG signals.

[0007] Therefore, it is essential to study a robust automatic sleep staging technology based on visual Transformer. Summary of the Invention

[0008] To address the limitations of feature representation performance, scarcity of reliable PSG datasets, and poor robustness of models to PSG signal noise and artifacts in sleep staging tasks, this invention introduces a visual Transformer-based encoder. By using transfer learning, it reduces the dependence on large amounts of PSG data. At the same time, a data augmentation module is designed to address noise and artifacts in PSG signals, thereby learning high-performance and highly robust feature representations.

[0009] This model incorporates three key ideas: designing a frame-level encoder based on a visual Transformer, employing a sliding window to capture short-term contextual information, and using GRU to achieve long-term sequence-level modeling; reducing the dependence on large-scale PSG datasets by using a pre-trained model on an out-of-domain dataset and fine-tuning it on the sleep PSG dataset through cross-modal transfer learning; and proposing a dynamic data augmentation module for the EEG and EOG channels, enabling the model to learn more robust feature representations.

[0010] The method of this invention is an automatic sleep staging method based on visual Transformer, and the specific steps are as follows:

[0011] Step 1: Introduce raw PSG signals from multiple channels at multiple times, and process the raw PSG signals from multiple times in each channel through a sliding window to obtain multiple PSG signal sequences for each channel;

[0012] Step 2: Obtain multiple sets of data-augmented PSG signal samples for each channel by performing data augmentation on multiple PSG signal sequences for each channel. Construct each set of PSG signal samples using the same set of data-augmented PSG signal samples from multiple channels, and manually label the actual sleep stage of each set of PSG signal samples.

[0013] Step 3: Construct a sleep staging network by cascading a visual Transformer frame-level encoder, a bidirectional GRU sequence-level encoder, and a softmax layer. Input each group of PSG signal samples into the sleep staging network to predict the predicted sleep stage of each group of PSG signal samples. Initialize the sleep staging network through cross-modal transfer learning. Construct a loss function model for the sleep staging network by combining the actual sleep stage of each group of PSG signal samples. Train the optimized sleep staging network using the ADAM optimizer.

[0014] Step 4: Real-time acquisition of PSG signals at multiple times. The real-time PSG signal samples are obtained through the sliding window processing in Step 1. The real-time PSG signal samples are then used to predict the real-time sleep stage through the optimized sleep staging network.

[0015] Preferably, the original PSG signal at multiple times for each channel in step 1 is defined as follows:

[0016] datac=(datac,1,datac,2,...,datac,L)

[0017] c∈[1, C]

[0018] Where datac represents the original PSG signal at multiple times in the c-th channel, datac,n represents the original PSG signal at the n-th time in the c-th channel, n∈[1,L], L represents the number of original times, and C represents the number of channels;

[0019] The window coverage area for the sliding window processing in step 1 is from (n-(T0-1) / 2) to (n+(T0-1) / 2);

[0020] The window length for the sliding window processing in step 1 is: T0;

[0021] The multiple PSG signal sequences for each channel mentioned in step 1 are specifically as follows:

[0022]

[0023]

[0024] c∈[1, C]

[0025] Among them, Sdata c S represents the PSG signal within a sliding window at multiple times in the c-th channel. c，i Let represent the PSG signal within the sliding window at time i of the c-th channel, where i ∈ [1, T1], T1 represents the number of PSG signal sequences, and C represents the number of channels;

[0026] Preferably, the data augmentation process in step 2 specifically includes:

[0027] Signal denoising, signal channel interference processing, signal additive noise processing, and signal masking frequency processing are all performed with data augmentation based on a certain random probability.

[0028] Step 2, which describes multiple sets of PSG signal samples after data enhancement processing for each channel, specifically includes:

[0029]

[0030] c∈[1, C]

[0031] Among them, Sdata′ c Let dS′ represent the data-enhanced PSG signal at multiple times in the c-th channel. c，m Let m represent the PSG signal sample after data augmentation processing of the m group of the c-th channel, where m∈[1, T1] and T1 represents the number of PSG signal samples after data augmentation processing;

[0032] Step 2 describes constructing each group of PSG signal samples from the same set of data augmentation processed across multiple channels, as follows:

[0033] S′ i =(dS′) 1，i dS′ 2，i ,...dS′ C，i )

[0034] i∈[1,T1]

[0035] Among them, S′ i Let T1 represent the i-th group of PSG signal samples, T1 represent the number of PSG signal samples, and C represent the number of channels;

[0036] As a preferred embodiment, the visual Transformer frame encoder described in step 3 is composed of a time-frequency transformation layer, a time-spectrum segmentation layer, a linear projection layer, a position coding layer, a multi-head attention layer, a fully connected layer, and a token connection layer cascaded in sequence.

[0037] The time-frequency transformation layer will convert the i-th group of PSG signal samples S′ i The short-time Fourier transform spectrum of the i-th group of PSG signal samples is calculated using the short-time Fourier transform, and is expressed as follows:

[0038] F i =(dF 1，i dF 2，i , ...dF C，i )

[0039] i∈[1,T1]

[0040] Among them, F i Let dF represent the short-time Fourier transform spectrum of the i-th PSG signal sample, T1 represent the number of PSG signal samples, C represent the number of channels, and dF represent the frequency response of the signal. c，i This represents the short-time Fourier transform spectrum of the i-th group of PSG signal samples in the c-th channel;

[0041] dF 1，i dF 2，i , ...dF G，i By splicing along the frequency axis, the spliced ​​time-frequency spectrum X of the i-th group of PSG signal samples is obtained. fft，i , for X fft，i The logarithmic time spectrum of the concatenated PSG signal samples of the i-th group is obtained by performing a logarithmic transformation. This logarithmic time spectrum is then normalized using a normal distribution method, yielding the normalized time spectrum X′ of the i-th PSG signal samples. fft，i ;

[0042] The time-spectrum block layer normalizes the time-spectrum X′ of the i-th group of PSG signal samples. fft，i The sequence is divided into N p×p patches, which represent the time-frequency spectrum of the i-th PSG signal sample after being divided into blocks, as follows:

[0043] X i =(x 1，i x 2，i , ..., x n，i , ..., x N，i )

[0044] n∈[1, N]

[0045] Where, x n，i This represents the nth patch in the time-spectrum after the i-th group of PSG signal samples is divided into blocks, where N is the total number of patches in the time-spectrum after the i-th group of PSG signal samples is divided into blocks.

[0046] The linear projection layer transforms each patch in the time spectrum of the i-th group of PSG signal samples into a patch vector sequence of the i-th group of PSG signal samples through linear projection, as specifically defined below:

[0047] E i =(E i，1 E i，2 , ..., E i，N )

[0048] Among them, E i，nThis represents the nth patch vector of the i-th group of PSG signal samples, where N is the total number of patches in the time spectrum after the i-th group of PSG signal samples is divided into blocks.

[0049] The positional encoding layer randomly superimposes the vectors of each patch and embeds them at different positions to obtain the encoded feature sequence of the i-th group of PSG signal samples, specifically defined as follows:

[0050]

[0051]

[0052] Among them, P i，n This represents the embedding position of the nth patch in the i-th group of PSG signal samples, where N is the total number of patches in the time-frequency spectrum after the i-th group of PSG signal samples is divided into blocks. This represents the encoded feature of the nth patch vector of the i-th group of PSG signal samples;

[0053] The input feature sequence of the Transformer for the i-th group of PSG signal samples is constructed as follows:

[0054]

[0055] in, This indicates the embedding of the [CLS] marker at the input of the i-th group of PSG signal samples. The learnable [CLS] label at the beginning of the sequence of the i-th group of PSG signal samples. This represents the encoded feature of the nth patch in the time spectrum after the i-th group of PSG signal samples is divided into blocks;

[0056] The multi-head attention layer and the fully connected layer will The output feature sequence of the i-th group of PSG signal samples is obtained by processing through a multi-layer Transformer encoder.

[0057]

[0058] in, This represents the output feature sequence of the i-th group of PSG signal samples. The [CLS] label represents the output of the i-th group of PSG signal samples. This represents the output feature of the nth patch of the i-th group of PSG signal samples;

[0059] Define the length of the target sleep frame as N0, and construct the output feature vector sequence of the i-th group of PSG signal samples as follows:

[0060]

[0061] The token connection layer will By concatenating with the average value of Di, we obtain the characteristics of a single sleep frame of the i-th group of PSG signal samples, defined as follows:

[0062]

[0063] in, This represents the output feature of the nth patch of the i-th group of PSG signal samples, and Concat represents concatenation;

[0064] Will Defined as a feature sequence of a single sleep frame;

[0065] Step 3 describes a bidirectional GRU sequence-level encoder that encodes the feature sequence of a single sleep frame. Convert to sequence-level feature vector sequence

[0066] Step 3, the softmax layer, will sequence the feature vectors. Mapped to the corresponding predicted sleep stage probability sequence, the predicted sleep stage probability sequence is defined as follows:

[0067] π i =(π) i，1 , π i，2 , ..., π i，K ) T

[0068] Where, π i，k This represents the probability that the i-th group of PSG signal samples is predicted to be in sleep stage k;

[0069] The sleep staging network loss function model described in step 3 is as follows:

[0070]

[0071] Among them, y i This represents the one-hot encoded vector of the true sleep stage of the i-th group of PSG signal samples.

[0072] To predict the probability sequence of sleep stages;

[0073] This invention addresses the problems of poor robustness to PSG noise and artifacts, limited feature representation capabilities, and scarcity of reliable PSG datasets in sleep staging tasks. It designs a data augmentation module based on PSG signal noise and artifacts to improve the model's robustness to these factors. Furthermore, it introduces a visual Transformer-based encoder to enhance the model's feature representation capabilities and employs transfer learning to reduce reliance on large amounts of PSG data. Attached Figure Description

[0074] Figure 1 : Overall structural diagram of an embodiment of the present invention;

[0075] Figure 2 The denoised waveforms above and below the EEG in this embodiment of the invention, and their time-spectrum diagrams;

[0076] Figure 3 The above and below waveforms of the EEG and the time spectrum after signal interference in this embodiment of the invention;

[0077] Figure 4 The above and below waveforms of the EEG and the time spectrum of the present invention after adding high-frequency noise;

[0078] Figure 5 The above and below waveforms of the EEG and the time spectrum of the present invention after adding low-frequency noise;

[0079] Figure 6 The above and below waveforms of the EEG and the time spectrum of the present invention after adding low-frequency noise;

[0080] Figure 7 : A schematic diagram of the cross-modal transfer learning implementation scheme according to an embodiment of the present invention;

[0081] Figure 8 : A schematic diagram of a bidirectional GRU-based sequence encoder according to an embodiment of the present invention; Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0084] The following is combined Figures 1 to 8 This invention introduces an automatic sleep staging method based on a visual Transformer, as detailed below:

[0085] Step 1: Introduce raw PSG signals from multiple channels at multiple times, and process the raw PSG signals from multiple times in each channel through a sliding window to obtain multiple PSG signal sequences for each channel;

[0086] The original PSG signal at multiple times for each channel mentioned in step 1 is defined as follows:

[0087] datac=(datac,1,datac,2,...,datac,L)

[0088] c∈[1, C]

[0089] Where datac represents the original PSG signal at multiple times in the c-th channel, and datac,n represents the original PSG signal at the n-th time in the c-th channel, n∈[1,L]:

[0090] The window coverage area for the sliding window processing in step 1 is from (n-(T0-1) / 2) to (n+(T0-1) / 2);

[0091] The window length for the sliding window processing in step 1 is: T0;

[0092] The multiple PSG signal sequences for each channel mentioned in step 1 are specifically as follows:

[0093]

[0094]

[0095] c∈[1, C]

[0096] Among them, Sdata c S represents the PSG signal within a sliding window at multiple times in the c-th channel. c，i Let represent the PSG signal within the sliding window at time i in the c-th channel, i∈[1,T1], T1 represents the number of PSG signal sequences, T1=21 represents the number of original time points, T0=3 represents the window length of the sliding window processing, and C=2 represents the number of channels;

[0097] Step 2: Obtain multiple sets of data-augmented PSG signal samples for each channel by performing data augmentation on multiple PSG signal sequences for each channel. Construct each set of PSG signal samples using the same set of data-augmented PSG signal samples from multiple channels, and manually label the actual sleep stage of each set of PSG signal samples.

[0098] The data augmentation process described in step 2 specifically includes:

[0099] Signal denoising, signal channel interference processing, signal additive noise processing, and signal masking frequency processing are all performed with data augmentation based on a certain random probability.

[0100] Step 2, which describes multiple sets of PSG signal samples after data enhancement processing for each channel, specifically includes:

[0101]

[0102] c∈[1, C]

[0103] Among them, Sdata′ c Let dS′ represent the data-enhanced PSG signal at multiple times in the c-th channel. c，m Let m represent the PSG signal sample after data augmentation processing of the m group of the c-th channel, where m∈[1, T1] and T1 represents the number of PSG signal samples after data augmentation processing;

[0104] Step 2 describes constructing each group of PSG signal samples from the same set of data augmentation processed across multiple channels, as follows:

[0105] S′ i =(dS) 1，i dS′ 2，i ,...dS′ C，i )

[0106] i∈[1,T1]

[0107] Among them, S′ i Let T1 represent the i-th group of PSG signal samples, T1 represent the number of PSG signal samples, and C represent the number of channels;

[0108] The signal denoising process described above: Low-pass and high-pass signals are of significant value in sleep-related research. This enhancement method uses band-pass filtering to reduce noise in the PSG signal. The signal is passed through a first-order Butterworth filter, retaining only in-band frequencies. The probability of band-pass filtering denoising being activated during training is 0.5.

[0109] like Figure 2As shown, the time-domain signals and time-spectrum of EEG (top waveform) and EOG (bottom waveform) after signal denoising processing;

[0110] The signal channel interference processing involves the following steps: Because F3 and F4 are quite close to the eye, the frontal leads will pick up eye movement artifacts, which can be seen as related deflections in the EEG signal. Specifically, these eye movement artifacts manifest as deflections in the EOG leads appearing in the frontal leads. Similarly, the EOG channel may sometimes receive signals from the EEG channel. This artifact is simulated by superimposing EEG and EOG signals at a specific ratio. During training, the probability of signal interference activation is 0.4, with both the probability of the EOG channel receiving an EEG signal and the probability of the EEG channel receiving an EOG signal being 50%.

[0111] like Figure 3 As shown, the time-domain signals and time-spectrum of EEG (top waveform) and EOG (bottom waveform) after signal channel interference processing;

[0112] The additive noise processing of the signal involves simulating potential slow-frequency artifacts and muscle artifacts by adding high-frequency low-amplitude or low-frequency high-amplitude noise to the EEG and EOG channels. Slow-frequency artifacts are typically caused by sweating or body movements related to breathing. Sweat alters the electrode potential, diluting the conductive medium between the electrode and the skin, producing a delta-wave-like artifact. Muscle artifacts are usually generated by localized muscle activity, with frequencies ranging from 20-200 Hz. These artifacts are simulated by adding independent, identically distributed high-frequency low-amplitude or low-frequency high-amplitude noise to the EEG and EOG channels. The probability of additive noise activation during training is 0.5, while the probability of adding high-frequency or low-frequency noise is 50%.

[0113] like Figure 4 , 5 As shown, the time-domain signals and time-spectrum of EEG (top waveform) and EOG (bottom waveform) after adding high-frequency noise and low-frequency noise respectively;

[0114] The signal masking frequency processing described: Masking techniques are widely used in audio and video research, but the effectiveness of masking strategies in the time spectrum of PSG signals has not yet been explored. In this enhancement module, a set of continuous frequency channels or time steps are masked by using frequency masking and time masking. Frequency masking is achieved by filtering the time signal with a band-stop filter, while time masking is achieved by setting consecutive sampling points to zero.

[0115] like Figure 6 As shown, the original time spectrum and the masked time spectrum of the EEG (top waveform) and EOG (bottom waveform) after frequency masking are displayed.

[0116] Step 3: Construct a sleep staging network by cascading a visual Transformer frame-level encoder, a bidirectional GRU sequence-level encoder, and a softmax layer. Input each group of PSG signal samples into the sleep staging network to predict the predicted sleep stage of each group of PSG signal samples. Initialize the sleep staging network through cross-modal transfer learning. Construct a loss function model for the sleep staging network by combining the actual sleep stage of each group of PSG signal samples. Train the optimized sleep staging network using the ADAM optimizer.

[0117] The visual Transformer frame encoder described in step 3 consists of a time-frequency transformation layer, a time-spectrum segmentation layer, a linear projection layer, a position coding layer, a multi-head attention layer, a fully connected layer, and a token connection layer, which are cascaded in sequence.

[0118] The time-frequency transformation layer will convert the i-th group of PSG signal samples S′ i The short-time Fourier transform spectrum of the i-th group of PSG signal samples is calculated using the short-time Fourier transform, and is expressed as follows:

[0119] F i =(dF 1，i dF 2，i , ...dF G，i )

[0120] i∈[1,T1]

[0121] Among them, F i Let dF represent the short-time Fourier transform spectrum of the i-th PSG signal sample, T1 represent the number of PSG signal samples, C represent the number of channels, and dF represent the frequency response of the signal. c，i This represents the short-time Fourier transform spectrum of the i-th group of PSG signal samples in the c-th channel;

[0122] dF 1，i dF 2，i , ...dF C，i By splicing along the frequency axis, the spliced ​​time-frequency spectrum X of the i-th group of PSG signal samples is obtained. fft，i , for X fft，i The logarithmic time spectrum of the concatenated PSG signal samples of the i-th group is obtained by performing a logarithmic transformation. This logarithmic time spectrum is then normalized using a normal distribution method, yielding the normalized time spectrum X′ of the i-th PSG signal samples. fft，i ;

[0123] The time-spectrum block layer normalizes the time-spectrum X′ of the i-th group of PSG signal samples. fft，i The sequence is divided into N p×p patches, which represent the time-frequency spectrum of the i-th PSG signal sample after being divided into blocks, as follows:

[0124] X i =(x 1，i x 2，i , ..., x n，i , ..., x N，i )

[0125] n∈[1, N]

[0126] Where, x n，i This represents the nth patch in the time-spectrum after the i-th group of PSG signal samples is divided into blocks, where N is the total number of patches in the time-spectrum after the i-th group of PSG signal samples is divided into blocks.

[0127] The linear projection layer transforms each patch in the time spectrum of the i-th group of PSG signal samples into a patch vector sequence of the i-th group of PSG signal samples through linear projection, as specifically defined below:

[0128] E i =(E i，1 E i，2 , ..., E i，N )

[0129] Among them, E i，n This represents the nth patch vector of the i-th group of PSG signal samples, where N is the total number of patches in the time spectrum after the i-th group of PSG signal samples is divided into blocks.

[0130] The positional encoding layer randomly superimposes the vectors of each patch and embeds them at different positions to obtain the encoded feature sequence of the i-th group of PSG signal samples, specifically defined as follows:

[0131]

[0132]

[0133] Among them, P i，n This represents the embedding position of the nth patch in the i-th group of PSG signal samples, where N is the total number of patches in the time-frequency spectrum after the i-th group of PSG signal samples is divided into blocks. This represents the encoded feature of the nth patch vector of the i-th group of PSG signal samples;

[0134] The input feature sequence of the Transformer for the i-th group of PSG signal samples is constructed as follows:

[0135]

[0136] in, This indicates the embedding of the [CLS] marker at the input of the i-th group of PSG signal samples.

[0137] The learnable [CLS] label at the beginning of the sequence of the i-th group of PSG signal samples. This represents the encoded feature of the nth patch in the time spectrum after the i-th group of PSG signal samples is divided into blocks;

[0138] The multi-head attention layer and the fully connected layer will The output feature sequence of the i-th group of PSG signal samples is obtained by processing through a multi-layer Transformer encoder.

[0139]

[0140] in, This represents the output feature sequence of the i-th group of PSG signal samples. The [CLS] label represents the output of the i-th group of PSG signal samples. This represents the output feature of the nth patch of the i-th group of PSG signal samples;

[0141] Define the length of the target sleep frame as N0, and construct the output feature vector sequence of the i-th group of PSG signal samples as follows:

[0142]

[0143] The token connection layer will By concatenating with the average value of Di, we obtain the characteristics of a single sleep frame of the i-th group of PSG signal samples, defined as follows:

[0144]

[0145] in, This represents the output feature of the nth patch of the i-th group of PSG signal samples, and Concat represents concatenation;

[0146] Step 3, which involves initializing the sleep staging network through cross-modal transfer learning, specifically involves: averaging channel weights and adaptive input length.

[0147] The channel weight averaging: The weights corresponding to the three input channels of the linear projection layer of the pre-trained Transformer are averaged and used as the weights of the linear projection layer of the visual Transformer frame encoder.

[0148] The input length is adaptive: Since the input shape of the pre-trained Transformer is fixed (224 × 224 or 384 × 384), this differs from typical PSG time-frequency maps. To address the inconsistency in position embedding length, a cutting and bilinear interpolation method is used for adaptive position embedding.

[0149] like Figure 7 The diagram shown is a schematic of a cross-modal transfer learning implementation scheme.

[0150] Will Defined as a feature sequence of a single sleep frame;

[0151] Step 3 describes a bidirectional GRU sequence-level encoder that encodes the feature sequence of a single sleep frame.

[0152] Convert to sequence-level feature vector sequence

[0153] like Figure 8 The diagram shown is a schematic of a sequence encoder based on a bidirectional GRU.

[0154] Step 3, the softmax layer, will sequence the feature vectors. Mapped to the corresponding predicted sleep stage probability sequence, the predicted sleep stage probability sequence is defined as follows:

[0155] π i =(π) i，1 , π i，2 , ..., π i，K ) τ

[0156] Where, π i，k This represents the probability that the i-th group of PSG signal samples is predicted to be in sleep stage k;

[0157] The sleep staging network loss function model described in step 3 is as follows:

[0158]

[0159] Among them, y i This represents the one-hot encoded vector of the true sleep stage of the i-th group of PSG signal samples;

[0160] Step 4: Wear the polysomnography device on the human body according to sleep medicine standards to collect PSG signals at multiple times in real time and transmit them to the computer. The computer will process the PSG signals collected at multiple times in real time through the sliding window process in Step 1 to obtain real-time PSG signal samples. The real-time PSG signal samples will be used to predict the real-time sleep stage through the optimized sleep staging network.

[0161] It should be understood that any parts not described in detail in this specification belong to the prior art.

[0162] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. An automatic sleep staging method based on visual Transformer, characterized in that, Includes the following steps: Step 1: Introduce raw PSG signals from multiple channels at multiple times, and process the raw PSG signals from multiple times in each channel through a sliding window to obtain multiple PSG signal sequences for each channel; Step 2: Obtain multiple sets of data-augmented PSG signal samples for each channel by performing data augmentation on multiple PSG signal sequences for each channel. Construct each set of PSG signal samples using the same set of data-augmented PSG signal samples from multiple channels, and manually label the actual sleep stage of each set of PSG signal samples. Step 3: Construct a sleep staging network by cascading a visual Transformer frame-level encoder, a bidirectional GRU sequence-level encoder, and a softmax layer. Input each group of PSG signal samples into the sleep staging network to predict the predicted sleep stage of each group of PSG signal samples. Initialize the sleep staging network through cross-modal transfer learning. Construct a loss function model for the sleep staging network by combining the actual sleep stage of each group of PSG signal samples. Train the optimized sleep staging network using the ADAM optimizer. Step 4: Real-time acquisition of PSG signals at multiple times. The real-time PSG signal samples are obtained through the sliding window processing in Step 1. The real-time PSG signal samples are then used to predict the real-time sleep stage through the optimized sleep staging network. Specifically, the data augmentation process described in step 2 is as follows: Signal denoising, signal channel interference processing, signal additive noise processing, and signal masking frequency processing are all performed with data augmentation based on a certain random probability. Step 2, which describes multiple sets of PSG signal samples after data enhancement processing for each channel, specifically includes: in, This represents the data-enhanced PSG signal at multiple times in the c-th channel. This represents the PSG signal sample after augmentation processing of the m-th data group in the c-th channel, where m∈[1, ..., ... ], Indicates the number of PSG signal samples; Step 2 describes constructing each group of PSG signal samples from the same set of data augmentation processed across multiple channels, as follows: i∈[1, ] in, This represents the i-th group of PSG signal samples, where C represents the number of channels; The visual Transformer frame encoder described in step 3 consists of a time-frequency transformation layer, a time-spectrum segmentation layer, a linear projection layer, a position coding layer, a multi-head attention layer, a fully connected layer, and a token connection layer, which are cascaded in sequence.

2. The automatic sleep staging method based on visual Transformer according to claim 1, characterized in that: The original PSG signal at multiple times for each channel mentioned in step 1 is defined as follows: datac=( datac,1, datac,2, ... , datac,L) c∈[1,C] Where datac represents the original PSG signal at multiple times in the c-th channel, datac,n represents the original PSG signal at the n-th time in the c-th channel, n∈[1,L], and L represents the number of original times; The window coverage area for the sliding window processing in step 1 is: arrive ; The window length for the sliding window processing in step 1 is: ; The multiple PSG signal sequences for each channel mentioned in step 1 are specifically as follows: ) =( ) in, This represents the PSG signal within a sliding window at multiple times in the c-th channel. This represents the PSG signal within the sliding window at time i in the c-th channel, where i ∈ [1, ..., ... ].

3. The automatic sleep staging method based on visual Transformer according to claim 1, characterized in that: Step 3, the time-frequency transformation layer, will convert the i-th group of PSG signal samples... The short-time Fourier transform spectrum of the i-th group of PSG signal samples is calculated using the short-time Fourier transform, and is expressed as follows: i∈[1, ] in, This represents the short-time Fourier transform spectrum of the i-th group of PSG signal samples. This represents the short-time Fourier transform spectrum of the i-th group of PSG signal samples in the c-th channel; Will By splicing along the frequency axis, the spliced ​​time spectrum of the i-th group of PSG signal samples is obtained. ,right A logarithmic transform is performed to obtain the logarithmic time spectrum of the concatenated PSG signal samples of the i-th group. The logarithmic time spectrum of the concatenated PSG signal samples of the i-th group is then normalized using a normal distribution method, yielding the normalized time spectrum of the i-th PSG signal samples. ; The time-spectrum segmentation layer normalizes the time-spectrum of the i-th group of PSG signal samples. Divide into N The size of the patch sequence represents the time-frequency spectrum of the i-th group of PSG signal samples after block division, as follows: n∈[1,N] in, This represents the nth patch in the time-spectrum after the i-th group of PSG signal samples is divided into blocks, where N is the total number of patches in the time-spectrum after the i-th group of PSG signal samples is divided into blocks. The linear projection layer transforms each patch in the time spectrum of the i-th group of PSG signal samples into a patch vector sequence of the i-th group of PSG signal samples through linear projection, as specifically defined below: in, This represents the nth patch vector of the i-th group of PSG signal samples; The positional encoding layer randomly superimposes the vectors of each patch and embeds them at different positions to obtain the encoded feature sequence of the i-th group of PSG signal samples, specifically defined as follows: =( , ,..., ) in, This indicates the embedding position of the nth patch in the i-th group of PSG signal samples. This represents the encoded feature of the nth patch vector of the i-th group of PSG signal samples; The input feature sequence of the Transformer for the i-th group of PSG signal samples is constructed as follows: in, This indicates the embedding of the [CLS] marker at the input of the i-th group of PSG signal samples. The learnable [CLS] label at the beginning of the sequence of the i-th group of PSG signal samples. This represents the encoded feature of the nth patch in the time spectrum after the i-th group of PSG signal samples is divided into blocks; The multi-head attention layer and the fully connected layer will The output feature sequence of the i-th group of PSG signal samples is obtained by processing through a multi-layer Transformer encoder. : in, This represents the output feature sequence of the i-th group of PSG signal samples. The [CLS] label represents the output of the i-th group of PSG signal samples. This represents the output feature of the nth patch of the i-th group of PSG signal samples; Define the length of the target sleep frame as Construct the output feature vector sequence of the i-th group of PSG signal samples, defined as follows: D i = The token connection layer will With D i The average values ​​of the signals are concatenated to obtain the features of a single sleep frame of the i-th group of PSG signal samples, defined as follows: in, This represents the output feature of the nth patch of the i-th group of PSG signal samples, and Concat indicates concatenation. Will Defined as a feature sequence of a single sleep frame.

4. The automatic sleep staging method based on visual Transformer according to claim 1, characterized in that: Step 3 describes a bidirectional GRU sequence-level encoder that encodes the feature sequence of a single sleep frame. Convert to sequence-level feature vector sequence ; Step 3, the softmax layer, will sequence the feature vectors. Mapped to the corresponding predicted sleep stage probability sequence, the predicted sleep stage probability sequence is defined as follows: in, This represents the probability that the i-th group of PSG signal samples is predicted to be in sleep stage k.

5. The automatic sleep staging method based on visual Transformer according to claim 1, characterized in that: The sleep staging network loss function model described in step 3 is as follows: in, This represents the one-hot encoded vector of the true sleep stage of the i-th group of PSG signal samples. To predict the probability sequence of sleep stages.

Images