Sleep staging processing method, apparatus, program product, and electronic device

Abstract

This disclosure provides a sleep staging processing method, apparatus, program product, and electronic device, relating to the field of data processing technology. The sleep staging processing method includes: acquiring multiple pairs of real EEG samples; identifying pairs of real EEG samples adjacent to sleep stages from the multiple pairs of real EEG samples, and generating virtual EEG sample pairs based on these adjacent pairs; and training a sleep staging model based on the virtual and real EEG sample pairs. In this disclosure, by constructing virtual EEG sample pairs that conform to physiological laws using pairs of real EEG samples adjacent to sleep stages, data augmentation is achieved. This avoids interference with the decision boundary of the sleep staging model caused by training with dirty data, and improves the discriminative ability of the sleep staging model in fuzzy states.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a sleep staging method, a sleep staging device, a computer program product, and an electronic device. Background Technology

[0002] EEG (electroencephalogram) records the electrical activity of neurons in the brain and plays an irreplaceable role in revealing sleep structure. However, training high-precision sleep staging models typically requires massive amounts of accurately labeled sample data. Manual annotation is not only time-consuming and labor-intensive, but also struggles to achieve ideal levels of consensus among experts (Kappa coefficient). Therefore, data augmentation has become a key approach to addressing the scarcity of labeled samples for sleep staging models and improving their robustness.

[0003] In related technologies, some commonly used data augmentation techniques (such as rotation, scaling, random cropping and splicing) can expand the number of samples, but they are divorced from the generation logic of real EEG data. This can easily lead to the generated samples violating physiological laws, making it difficult to reflect the characteristics of EEG data in real scenarios. This can easily interfere with the decision boundary of the sleep staging model, thereby affecting the ability of the sleep staging model to make judgments in fuzzy states.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a sleep staging method, a sleep staging device, a computer program product, and an electronic device to at least partially solve the problem that data-augmented samples violate physiological laws in related technologies.

[0006] According to a first aspect of this disclosure, a sleep staging method is provided, the method comprising: acquiring multiple pairs of real EEG samples, each pair including real EEG samples and a real sample label representing the sleep stage to which the real EEG samples belong; determining pairs of real EEG samples adjacent to each sleep stage from the multiple pairs of real EEG samples, and generating virtual EEG sample pairs based on the pairs of real EEG samples adjacent to each sleep stage; each virtual EEG sample pair including virtual EEG samples and a virtual sample label representing the sleep stage to which the virtual EEG samples belong; and training a sleep staging model based on the virtual EEG sample pairs and the real EEG sample pairs, the sleep staging model being used for sleep stage classification.

[0007] According to a second aspect of this disclosure, a sleep staging processing apparatus is provided, the apparatus comprising: a real sample pair acquisition module, configured to acquire multiple real EEG sample pairs, the real EEG sample pairs including real EEG samples and real sample labels for characterizing the sleep stage to which the real EEG samples belong; a virtual sample pair generation module, configured to determine adjacent real EEG sample pairs for sleep stages from the multiple real EEG sample pairs, and generate virtual EEG sample pairs based on the adjacent real EEG sample pairs for sleep stages; the virtual EEG sample pairs including virtual EEG samples and virtual sample labels for characterizing the sleep stage to which the virtual EEG samples belong; and a sleep staging model training module, configured to train a sleep staging model based on the virtual EEG sample pairs and the real EEG sample pairs, the sleep staging model being used for sleep stage classification.

[0008] According to a third aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the sleep staging method of the first aspect and its possible implementations.

[0009] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the sleep staging method of the first aspect and possible implementations thereof by executing the executable instructions.

[0010] The technical solution disclosed herein has the following beneficial effects: In the aforementioned sleep staging process, multiple pairs of real EEG samples are acquired. Each pair includes a real EEG sample and a real sample label representing the sleep stage to which the real EEG sample belongs. From these multiple pairs of real EEG samples, pairs of real EEG samples adjacent to each sleep stage are identified, and virtual EEG sample pairs are generated based on these adjacent pairs. Each virtual EEG sample pair includes a virtual EEG sample and a virtual sample label representing the sleep stage to which the virtual EEG sample belongs. The sleep staging model is trained based on both the virtual and real EEG sample pairs and is used for sleep stage classification. On one hand, data augmentation based on adjacent real EEG sample pairs allows for the generation of high-quality, transitional virtual EEG sample pairs that conform to physiological principles, making them well-suited for sleep staging model training. On the other hand, training the model based on these physiologically consistent virtual EEG sample pairs avoids interference with the decision boundaries of the sleep staging model caused by training with dirty data, thus improving the model's ability to discriminate under ambiguous conditions. Attached Figure Description

[0011] Figure 1 This diagram illustrates a sleep staging method according to an exemplary embodiment. Figure 2 A schematic diagram illustrating a sleep stage transition constraint relationship in this exemplary embodiment is shown. Figure 3 A schematic diagram of a time-frequency dual-domain mask is shown in this exemplary embodiment; Figure 4 This diagram illustrates a flowchart of model training in this exemplary embodiment; Figure 5 This diagram illustrates a structural block diagram of a sleep staging device according to an exemplary embodiment of the present invention. Figure 6 An electronic device for implementing the above-described sleep staging method is shown in this exemplary embodiment. Detailed Implementation

[0012] Exemplary embodiments of this disclosure will be described more fully below with reference to the accompanying drawings.

[0013] The accompanying drawings are schematic illustrations of this disclosure and are not necessarily drawn to scale. Some block diagrams shown in the drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in hardware modules or integrated circuits, or in networks, processors, or microcontrollers. Implementations can be carried out in various forms and should not be construed as limited to the examples set forth herein. The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough description of embodiments of this disclosure. However, those skilled in the art will recognize that one or more specific details may be omitted when implementing the technical solutions of this disclosure, or other methods, components, apparatuses, steps, etc., may be used to replace one or more specific details.

[0014] In related technologies, some commonly used data augmentation techniques (such as rotation, scaling, random cropping and splicing) can expand the number of samples, but they are divorced from the generation logic of real EEG data. This can easily lead to the generated samples violating physiological laws, making it difficult to reflect the characteristics of EEG data in real scenarios. This can easily interfere with the decision boundary of the sleep staging model, thereby affecting the ability of the sleep staging model to make judgments in fuzzy states.

[0015] In view of the above problems, exemplary embodiments of this disclosure provide a sleep staging method, a sleep staging apparatus, a computer program product, and an electronic device.

[0016] In one alternative implementation, refer to Figure 1The diagram illustrates a sleep staging method, specifically including the following steps S110 to S130: Step S110: Obtain multiple pairs of real EEG samples. Each pair of real EEG samples includes real EEG samples and real sample labels used to characterize the sleep stage to which the real EEG samples belong. Step S120: From multiple real EEG sample pairs, determine the real EEG sample pairs that are adjacent to the sleep stage, and generate virtual EEG sample pairs based on the real EEG sample pairs that are adjacent to the sleep stage; the virtual EEG sample pair includes virtual EEG sample and virtual sample label used to characterize the sleep stage to which the virtual EEG sample belongs. Step S130: The sleep staging model is trained based on virtual EEG sample pairs and real EEG sample pairs. The sleep staging model is used to classify sleep stages.

[0017] Figure 1 In the steps shown, data augmentation is achieved based on real EEG sample pairs adjacent to sleep stages, resulting in high-quality virtual EEG sample pairs that conform to physiological laws and are well adapted to the training scenario of sleep staging model. Model training based on virtual EEG sample pairs that conform to physiological laws can avoid interference with the decision boundary of the sleep staging model caused by training with dirty data and can improve the discrimination ability of the sleep staging model in fuzzy states.

[0018] The following is about Figure 1 Each step in the process will be explained in detail.

[0019] In step S110, multiple pairs of real brainwave samples are acquired. Each pair of real brainwave samples includes a real brainwave sample and a real sample label used to characterize the sleep stage to which the real brainwave sample belongs.

[0020] Among them, real brainwave samples can be electroencephalogram (EEG) data obtained by collecting brainwave signals from one or more subjects using an EEG acquisition device.

[0021] Optionally, after the subject's brainwave signals are collected by an EEG acquisition device (such as a "multi-channel EEG signal acquisition device"), the collected brainwave signals can be processed by filtering and noise reduction, data segmentation, etc., to obtain high-quality real brainwave samples.

[0022] For example, a "multi-channel EEG signal acquisition device" can be used to simultaneously acquire EEG signals from multiple different regions of the brain and integrate them into a multi-channel data format to obtain multi-channel EEG data. After acquiring the multi-channel EEG data, all channels can be filtered from 0.1 to 70 Hz and notch filtered at 50 Hz. The multi-channel EEG data can be segmented according to a preset time window (e.g., 30 seconds) to obtain high-quality real EEG samples.

[0023] Among them, the real sample label of the sleep stage to which the real EEG sample belongs refers to the high-quality sleep stage category label obtained by qualified professionals through manual analysis, judgment and labeling of real EEG samples according to preset sleep stage standards.

[0024] The sleep stages may include, but are not limited to, wakefulness (WAKE), early light sleep (N1), middle light sleep (N2), deep sleep (N3), and rapid eye movement (REM) sleep, etc. This disclosure does not specifically limit them.

[0025] A real brainwave sample and its corresponding real sample label can constitute a real brainwave sample pair. For example: real brainwave sample and real brainwave samples Corresponding real sample labels It can form real brainwave sample pairs ( , ).

[0026] After obtaining multiple pairs of real EEG samples, step S120 can be executed to achieve sample enhancement.

[0027] In step S120, from multiple real EEG sample pairs, adjacent real EEG sample pairs of sleep stages are determined, and virtual EEG sample pairs are generated based on adjacent real EEG sample pairs of sleep stages; the virtual EEG sample pairs include virtual EEG samples and virtual sample labels used to characterize the sleep stage to which the virtual EEG samples belong.

[0028] Among them, the real EEG sample pairs that are adjacent during sleep stages refer to real EEG sample pairs that conform to the physiological adjacent rules during sleep stages.

[0029] Optionally, multiple pairs of real EEG samples can be traversed to find pairs of real EEG samples that are adjacent to each other during the sleep stage for later use.

[0030] For example, it can be determined whether two real EEG sample pairs are adjacent real EEG samples during the sleep stage based on the real sample labels in any two real EEG sample pairs.

[0031] In one optional implementation, the above-mentioned determination of adjacent real brainwave sample pairs from multiple real brainwave sample pairs can be achieved through the following steps: determining adjacent real brainwave sample pairs from multiple real brainwave sample pairs based on sleep stage transition constraint relationships.

[0032] The sleep stage transition constraint relationship includes the pre-configured ability to transition between different sleep stages. Specifically, the sleep stage transition constraint relationship can be configured based on physiological common sense. For example... Figure 2 As shown, a schematic diagram of sleep stage transition constraints is provided. The diagram uses lines to represent the transition capabilities between different sleep stages. It should be noted that... Figure 2 The transfer relationships described herein are merely illustrative. In practical applications, these transfer constraints can be further refined or adjusted according to the actual situation. This disclosure does not impose any specific limitations on them.

[0033] For example, real brainwave samples The real sample was labeled as being awake (WAKE), and the real EEG samples were labeled as being in the wakeful state. The real sample was labeled as early stage of light sleep (N1), and the real EEG samples were labeled as early stage of light sleep. The true sample was labeled as deep sleep (N3). In the sleep stage transition constraint relationship, wakefulness (WAKE) can directly transition to early light sleep (N1), but not directly to deep sleep (N3). Therefore, it can be considered that the true EEG sample... Comparison with real EEG samples Adjacent, real EEG sample pairs Comparison with real EEG samples Not adjacent.

[0034] By identifying pairs of real EEG samples adjacent to each sleep stage, a physiologically sound data basis is provided for sample enhancement. In one implementation, any two pairs of real EEG samples adjacent to each sleep stage can be linearly interpolated to generate a virtual EEG sample pair.

[0035] Among them, virtual EEG samples are non-real EEG data obtained through sample augmentation techniques.

[0036] Among them, the virtual sample label of the sleep stage to which the virtual EEG sample belongs is the sleep stage category label corresponding to the virtual EEG sample obtained through sample augmentation.

[0037] A virtual brainwave sample and its corresponding virtual sample label can constitute a virtual brainwave sample pair. For example, virtual brainwave samples and virtual brainwave samples Corresponding virtual sample labels It can form virtual brainwave sample pairs ( , ).

[0038] In one optional implementation, the above-mentioned generation of virtual EEG sample pairs based on adjacent real EEG sample pairs during sleep stages can be achieved through the following steps: obtaining interpolation weight coefficients; wherein, the interpolation weight coefficients are determined based on a preset probability distribution; and based on the interpolation weight coefficients, linearly interpolating adjacent real EEG sample pairs during sleep stages to obtain virtual EEG sample pairs.

[0039] The interpolation weighting coefficient can be used to quantify the contribution of adjacent real EEG samples from two sleep stages to each other. Specifically, the interpolation weighting coefficient can be determined based on a preset probability distribution, and the coefficient can take values ​​in the range of [0,1].

[0040] The preset probability distribution can be a probability distribution pre-specified by the developers, providing a data basis for determining the interpolation weight coefficients. For example, the preset probability distribution can be a Beta distribution.

[0041] Specifically, interpolation weight coefficients can be obtained by sampling from a preset probability distribution, and virtual EEG sample pairs can be obtained by linear interpolation of adjacent real EEG sample pairs during the sleep stage based on the interpolation weight coefficients.

[0042] For example, ( )and( A virtual EEG sample pair can be obtained by linear interpolation of two adjacent real EEG sample pairs from two sleep stages. , ),in, , , These are the interpolation weighting coefficients.

[0043] In the above steps, the sample augmentation method based on the real EEG sample pairs adjacent to the sleep stage simulates the transition state signal during the sleep stage transition process, which can effectively expand the decision boundary data of the sleep staging model.

[0044] In one optional implementation, the process of determining adjacent real EEG sample pairs from multiple real EEG sample pairs and generating virtual EEG sample pairs based on these pairs can be achieved through the following steps: determining adjacent real EEG sample pairs from multiple real EEG sample pairs based on sleep stage transition constraints; determining interpolation weight coefficients based on a preset probability distribution; and performing linear interpolation on the adjacent real EEG sample pairs based on the interpolation weight coefficients to obtain virtual EEG sample pairs.

[0045] By introducing a sleep stage transition matrix as a constraint, mixed interpolation is only allowed between physiologically adjacent sleep stage samples, which can generate transition state enhancement sample pairs that conform to physiological laws, namely the virtual EEG sample pairs in this disclosure.

[0046] In step S130, the sleep staging model is trained based on virtual EEG sample pairs and real EEG sample pairs. The sleep staging model is used to classify sleep stages.

[0047] Among them, the sleep staging model refers to an intelligent algorithm model that can automatically determine the current sleep stage of the EEG data to be processed. Its core function is to transform the EEG data to be processed into a clear sleep stage classification result.

[0048] Optionally, the sleep staging model may include, but is not limited to, an encoder and a classifier. The encoder can be used to extract and encode features from the input EEG data, while the classifier is used to classify and determine sleep stages based on the feature representation output by the encoder.

[0049] In an optional implementation, the training of the sleep staging model based on virtual EEG sample pairs and real EEG sample pairs can be achieved through the following steps: Virtual EEG samples from the virtual EEG sample pairs and real EEG samples from the real EEG sample pairs are respectively processed by the encoder in the sleep staging model for feature extraction and encoding, resulting in feature representations of the virtual EEG samples and real EEG samples; the feature representations of the virtual EEG samples and real EEG samples are respectively processed by the classifier in the sleep staging model for sleep stage classification, resulting in sleep stage prediction results for the virtual EEG samples and real EEG samples; the model parameters of the sleep staging model are updated based on a first difference and a second difference; wherein the first difference is the difference between the sleep stage prediction result of the virtual EEG samples and the virtual sample label of the virtual EEG samples; and the second difference is the difference between the sleep stage prediction result of the real EEG samples and the real sample label of the real EEG samples.

[0050] Specifically, the virtual EEG samples in the virtual EEG sample pair can be feature-extracted and encoded using the encoder in the sleep staging model to obtain the feature representation of the virtual EEG sample; the real EEG samples in the real EEG sample pair can be feature-extracted and encoded using the encoder in the sleep staging model to obtain the feature representation of the real EEG sample.

[0051] Specifically, the features of virtual EEG samples can be represented and classified into sleep stages using a classifier in a sleep staging model to obtain the sleep stage prediction results for virtual EEG samples; the features of real EEG samples can be represented and classified into sleep stages using a classifier in a sleep staging model to obtain the sleep stage prediction results for real EEG samples.

[0052] Optionally, the first difference and the second difference can be characterized by cross-entropy loss / classification loss.

[0053] For example, the cross-entropy loss between the sleep stage prediction result of the virtual EEG sample and the virtual sample label of the virtual EEG sample can be calculated, and the model parameters can be optimized through backpropagation, thereby achieving the purpose of training the model.

[0054] For example, the cross-entropy loss between the sleep stage prediction results of real EEG samples and the real sample labels of real EEG samples can be calculated, and parameters can be updated through backpropagation to train the model.

[0055] By combining real EEG sample pairs with virtual EEG sample pairs that conform to physiological laws, the sleep staging model can be trained. This can solve the problem of scarce sample labels in the sleep staging task to a certain extent, and at the same time avoid the sleep staging model learning incorrect classification boundaries.

[0056] Optionally, the trained sleep staging model can be either an initialized model without parameter configuration or a pre-configured model with pre-loaded reasonable parameters; this disclosure does not impose any specific limitations on this.

[0057] In an alternative implementation, the following steps may also be performed: obtaining a pre-trained target network; wherein the target network is an encoder-decoder structure network used to reconstruct the time-frequency spectrum of brain waves; and determining the parameters corresponding to the encoder in the sleep staging model based on the parameters corresponding to the encoder in the target network.

[0058] The target network is an encoder-decoder structure network capable of reconstructing incomplete time-frequency spectra.

[0059] Specifically, the parameters corresponding to the encoder in the target network can be transferred to the encoder in the sleep staging model to achieve parameter pre-configuration of the encoder in the sleep staging model.

[0060] Since the target network can reconstruct incomplete time-frequency spectra, the encoder in the sleep staging model can retain the inherent patterns of EEG during feature extraction by reusing the parameters corresponding to the encoder in the target network. This ensures that even with only a small amount of sample labeled data, the sleep staging model can extract high-quality feature representations, thereby improving the accuracy of sleep staging decisions.

[0061] In an optional implementation, the following steps may also be performed: acquiring multiple unlabeled EEG samples and converting them into time-frequency spectra to obtain original time-frequency spectra corresponding to the multiple unlabeled EEG samples; masking the original time-frequency spectra corresponding to the multiple unlabeled EEG samples in the time domain and / or frequency domain to obtain incomplete time-frequency spectra corresponding to the multiple unlabeled EEG samples; reconstructing the incomplete time-frequency spectra corresponding to the multiple unlabeled EEG samples through a target network to obtain reconstructed time-frequency spectra corresponding to the multiple unlabeled EEG samples; and training the target network based on the original time-frequency spectra and the reconstructed time-frequency spectra corresponding to the multiple unlabeled EEG samples.

[0062] Among them, unlabeled EEG samples refer to data samples that contain EEG data but lack labels for the corresponding sleep stage. They are characterized by large data volume and easy accessibility.

[0063] Optionally, unlabeled EEG samples can be EEG data obtained by collecting EEG signals from one or more subjects using an EEG acquisition device.

[0064] Optionally, after collecting brainwave signals from the subject using an EEG acquisition device (such as a "multi-channel EEG signal acquisition device"), the collected EEG signals can be processed by filtering and denoising, data segmentation, etc., to obtain high-quality unlabeled EEG samples.

[0065] For example, a "multi-channel EEG signal acquisition device" can be used to simultaneously acquire EEG signals from multiple different regions of the brain and integrate them into a multi-channel data format to obtain multi-channel EEG data. After acquiring the multi-channel EEG data, all channels can be filtered from 0.1 to 70 Hz and notch filtered at 50 Hz. The multi-channel EEG data can be segmented according to a preset duration (e.g., 30 seconds) as a segmented time window to obtain unlabeled EEG samples.

[0066] Among them, the original time-frequency map refers to the map obtained by converting unlabeled EEG samples into a time-frequency map.

[0067] Optionally, after acquiring unlabeled EEG samples, the samples can be converted into time-frequency spectra using short-time Fourier transform (STFT) to obtain the corresponding original time-frequency spectra. .in, This is the data representation of the initial time-frequency spectrum. express The dimensions involved are as follows: m represents the number of channels, T represents the time dimension, and F represents the frequency dimension.

[0068] In this context, an incomplete time-frequency spectrum refers to a time-frequency spectrum that is missing in the time and / or frequency domains. Specifically, the original time-frequency spectrum can be masked in the time and / or frequency domains to obtain an incomplete time-frequency spectrum. For example, the original time-frequency spectrum can be randomly masked in the time and / or frequency domains to obtain an incomplete time-frequency spectrum.

[0069] like Figure 3 The diagram illustrates a time-frequency dual-domain masking method. A preset time-frequency masking strategy is used to destroy the original time-frequency spectrum using a dual-domain masking generator, resulting in a fragmented time-frequency spectrum. The preset time-frequency masking strategy refers to masking rules pre-defined to meet the destruction requirements of the original time-frequency spectrum. These rules may include mask length, masking timing, and mask strength, but this disclosure does not specifically limit their application.

[0070] For example, when masking the original time-frequency spectrum in the time domain, multiple time steps (e.g., 15% of the total duration) can be randomly selected, and the values ​​of all frequencies in those steps can be set to zero to obtain a partial time-frequency spectrum. Similarly, when masking the original time-frequency spectrum in the frequency domain, multiple frequency bands (e.g., 15% of the total bandwidth) can be randomly selected, and the values ​​of all time points in those bands can be set to zero to obtain a partial time-frequency spectrum.

[0071] Reconstructing the time-frequency map refers to the time-frequency map obtained by completing the incomplete time-frequency map. Specifically, the incomplete time-frequency map that has been destroyed by masking can be reconstructed through the target network to obtain the reconstructed time-frequency map.

[0072] Optionally, the target network can be trained based on the difference between the original time-frequency map and the reconstructed time-frequency map, i.e., the reconstruction loss, to optimize the parameters of the encoder in the target network, so that the target network learns how to supplement reasonable EEG features for the missing parts of the incomplete time-frequency map.

[0073] For example, the training objective of the target network could be to minimize the mean square error between the reconstructed time-frequency map and the original time-frequency map.

[0074] When reconstructing the incomplete time-frequency spectrum masked in the time domain, the target network can infer the missing waveforms based on the context to learn temporal continuity.

[0075] When reconstructing the incomplete time-frequency spectrum masked in the frequency domain, the target network can use information from other frequency bands to infer the missing frequency bands in order to learn frequency domain coupling.

[0076] After the target network learns the global time-frequency dependence of brain waves through self-supervised reconstruction training using a large number of unlabeled EEG samples, the parameters corresponding to its encoder are transferred to the encoder in the sleep staging model. This allows the sleep staging model to achieve high-precision sleep staging results through fine-tuning training with a small number of labeled EEG samples, which greatly reduces the implementation cost.

[0077] In one optional implementation, after determining the parameters corresponding to the encoder in the sleep staging model based on the parameters corresponding to the encoder in the target network, the following steps can also be performed: determining the frozen parameters from the parameters corresponding to the encoder in the sleep staging model based on a preset parameter freezing strategy; controlling the frozen parameters not to be updated following the training process of the sleep staging model.

[0078] The preset parameter freezing strategy refers to a series of pre-defined rules used to control the parameter freezing behavior of the encoder in the sleep staging model. For example, for different network layers of the encoder in the sleep staging model (such as shallow feature extraction layers, mid-layer feature fusion layers, and deep feature mapping layers), the freezing initiation timing, freezing duration, and thawing trigger conditions are set separately. This strategy can be pre-configured by technical personnel based on actual application scenario requirements, model performance indicators, or historical experimental data; this disclosure does not impose specific limitations on it.

[0079] Among them, the frozen parameters will not be updated or adjusted with the input of training data during subsequent model iterations and data training, which can avoid the loss of the ability to "preserve the inherent laws of brain waves during feature extraction" to a certain extent.

[0080] like Figure 4 As shown, a flowchart for model training is provided, which may include stage 1 (target network training stage) and stage 2 (sleep staging model training stage).

[0081] In Stage 1: First, unlabeled EEG samples are transformed using time-frequency conversion to obtain the original time-frequency spectrum. Next, the original time-frequency spectrum is processed using a time-frequency dual-domain mask to obtain an incomplete time-frequency spectrum. Then, the incomplete time-frequency spectrum is processed by the encoder in the target network for feature extraction and encoding to obtain a feature representation of the incomplete time-frequency spectrum. Next, the feature representation of the incomplete time-frequency spectrum is used by the decoder to reconstruct the time-frequency spectrum, obtaining the reconstructed time-frequency spectrum. Finally, the reconstruction loss is calculated based on the original and reconstructed time-frequency spectra, and the parameters of the encoder in the target network are optimized based on the reconstruction loss. After the target network is trained, the parameters corresponding to the encoder in the target network can be transferred to the encoder in the sleep staging model.

[0082] In stage 2: First, the original time-frequency spectrum is obtained by time-frequency transformation using real EEG sample pairs. Then, the original time-frequency spectrum is enhanced with periodic perception (i.e., linear interpolation of adjacent real EEG sample pairs for sleep stages) to obtain virtual EEG sample pairs, which form a mixed sample pair set with the real EEG sample pairs. Next, the samples in the mixed sample pair set are used for feature extraction and encoding by the encoder in the sleep staging model, and the sleep stages are classified by the classifier to obtain the sleep stage prediction results. The classification loss is calculated and the parameters of the sleep staging model are updated by backpropagating the classification loss.

[0083] By using a large number of unlabeled EEG samples to perform self-supervised reconstruction training on the target network, and then transferring the parameters of its encoder to the encoder in the sleep staging model, the sleep staging model can achieve high-precision sleep staging results through fine-tuning training with a small number of labeled EEG samples, which greatly reduces the implementation cost.

[0084] Exemplary embodiments of this disclosure also provide a sleep staging processing apparatus. (See reference...) Figure 5 As shown, the sleep staging processing device 500 may include the following program modules: The real sample pair acquisition module 510 is used to acquire multiple real brainwave sample pairs. The real brainwave sample pairs include real brainwave samples and real sample labels used to characterize the sleep stage to which the real brainwave samples belong. The virtual sample pair generation module 520 is used to determine adjacent real brainwave sample pairs from multiple real brainwave sample pairs, and generate virtual brainwave sample pairs based on the adjacent real brainwave sample pairs of sleep stages; the virtual brainwave sample pair includes virtual brainwave samples and virtual sample labels used to characterize the sleep stage to which the virtual brainwave samples belong. The sleep staging model training module 530 is used to train the sleep staging model based on virtual EEG sample pairs and real EEG sample pairs. The sleep staging model is used to classify sleep stages.

[0085] In an optional implementation, based on the aforementioned scheme, the virtual sample pair generation module 520 determines adjacent real brainwave sample pairs from multiple real brainwave sample pairs, including: determining adjacent real brainwave sample pairs from multiple real brainwave sample pairs based on sleep stage transfer constraint relationships; wherein, the sleep stage transfer constraint relationships include pre-configured transfer capabilities between different sleep stages.

[0086] In an optional implementation, based on the aforementioned scheme, the virtual sample pair generation module 520 generates virtual EEG sample pairs based on adjacent real EEG sample pairs during sleep stages, including: obtaining interpolation weight coefficients; wherein the interpolation weight coefficients are determined based on a preset probability distribution; and based on the interpolation weight coefficients, linearly interpolating adjacent real EEG sample pairs during sleep stages to obtain virtual EEG sample pairs.

[0087] In an optional implementation, based on the aforementioned scheme, the sleep staging model includes an encoder and a classifier. The sleep staging model training module 530 can be configured to: extract and encode features from virtual EEG samples in virtual EEG sample pairs and real EEG samples in real EEG sample pairs using the encoder in the sleep staging model, respectively, to obtain feature representations of virtual EEG samples and real EEG samples; classify sleep stages using the classifier in the sleep staging model using the feature representations of virtual EEG samples and real EEG samples, respectively, to obtain sleep stage prediction results for virtual EEG samples and real EEG samples; update the model parameters of the sleep staging model based on a first difference and a second difference; wherein, the first difference is the difference between the sleep stage prediction result of the virtual EEG sample and the virtual sample label of the virtual EEG sample; and the second difference is the difference between the sleep stage prediction result of the real EEG sample and the real sample label of the real EEG sample.

[0088] In an optional implementation, based on the aforementioned scheme, the sleep staging processing device 500 further includes: a parameter transfer module for acquiring a pre-trained target network; wherein the target network is an encoder-decoder structure network for reconstructing the EEG time-frequency spectrum; and the parameters corresponding to the encoder in the sleep staging model are determined based on the parameters corresponding to the encoder in the target network.

[0089] In an optional implementation, based on the aforementioned scheme, the sleep staging processing device 500 further includes a target network training module. This target network training module can be configured to: acquire multiple unlabeled EEG samples and convert them into time-frequency spectra to obtain original time-frequency spectra corresponding to the multiple unlabeled EEG samples; perform masking processing on the original time-frequency spectra corresponding to the multiple unlabeled EEG samples in the time domain and / or frequency domain to obtain incomplete time-frequency spectra corresponding to the multiple unlabeled EEG samples; reconstruct the incomplete time-frequency spectra corresponding to the multiple unlabeled EEG samples using the target network to obtain reconstructed time-frequency spectra corresponding to the multiple unlabeled EEG samples; and train the target network based on the original time-frequency spectra and the reconstructed time-frequency spectra corresponding to the multiple unlabeled EEG samples.

[0090] In an optional implementation, based on the aforementioned scheme, after determining the parameters corresponding to the encoder in the sleep staging model based on the parameters corresponding to the encoder in the target network, the sleep staging processing device 500 further includes: a parameter freezing module, used to determine frozen parameters from the parameters corresponding to the encoder in the sleep staging model based on a preset parameter freezing strategy; and to control the frozen parameters not to be updated following the training process of the sleep staging model.

[0091] The specific details of each part of the above-mentioned device have been described in detail in the method section of the implementation plan. For any undisclosed details, please refer to the implementation plan of the method section, and therefore will not be repeated here.

[0092] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0093] An exemplary embodiment of this disclosure also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the above-described sleep staging processing method.

[0094] In one implementation, the computer program product can be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. The readable storage medium can be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. For example, the computer program product can be implemented as a non-volatile storage medium storing a computer program, such as read-only memory, NAND flash memory, etc.

[0095] In one implementation, the computer program product can be an intangible product containing a computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, installation package, or other digital file storing the computer program.

[0096] Computer program code can be written in one or more programming languages. Examples of programming languages ​​include C, Java, and C++. Program code can execute entirely on the user's computing device, partially on the user's computing device, or as a standalone software package. It can also execute partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via an internet connection provided by a mobile network operator).

[0097] Computer programs can be carried or transmitted via signals such as electricity, magnetism, light, electromagnetic radiation, and infrared rays. Electronic devices can convert signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, the processor of the electronic device to execute) the method steps of various exemplary embodiments of this disclosure, such as the sleep staging processing method described above, which includes the following steps: Multiple pairs of real EEG samples were acquired. Each pair of real EEG samples included real EEG samples and real sample labels used to characterize the sleep stage to which the real EEG samples belonged. From multiple real EEG sample pairs, identify real EEG sample pairs that are adjacent to each sleep stage, and generate virtual EEG sample pairs based on the real EEG sample pairs that are adjacent to each sleep stage; the virtual EEG sample pairs include virtual EEG samples and virtual sample labels used to characterize the sleep stage to which the virtual EEG samples belong. The sleep staging model is trained based on virtual and real EEG sample pairs and is used to classify sleep stages.

[0098] The above method and steps, based on adjacent real EEG sample pairs in sleep stages, achieve data augmentation and obtain high-quality virtual EEG sample pairs that conform to physiological laws, which can be well adapted to the training scenario of sleep staging model. Model training based on virtual EEG sample pairs that conform to physiological laws can avoid interference with the decision boundary of sleep staging model caused by training with dirty data, and can improve the discrimination ability of sleep staging model in ambiguous states.

[0099] An exemplary embodiment of this disclosure also provides an electronic device capable of implementing the above-described sleep staging method. The electronic device may include a processor and a memory. The memory stores executable instructions for the processor, such as program code. The processor executes the executable instructions to perform the method of this exemplary embodiment.

[0100] The following is for reference. Figure 6 The electronic device is illustrated by way of a general-purpose computing device. It should be understood that... Figure 6 The electronic device 600 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0101] like Figure 6 As shown, the electronic device 600 may include: a processor 610, a memory 620, a bus 630, an I / O (input / output) interface 640, and a network adapter 650.

[0102] Memory 620 may include volatile memory, such as RAM 621 and cache unit 622, and may also include non-volatile memory, such as ROM 623. Memory 620 may also include one or more program modules 624, such program modules 624 including, but not limited to: operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. For example, program module 624 may include the modules in the above-described device.

[0103] The processor 610 may include one or more processing units, such as an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor, and / or an NPU (Neural-Network Processing Unit).

[0104] The processor 610 can be used to execute executable instructions stored in the memory 620, such as performing any one or more method steps in this exemplary embodiment.

[0105] For example, processor 610 may perform the following steps: Multiple pairs of real EEG samples were acquired. Each pair of real EEG samples included real EEG samples and real sample labels used to characterize the sleep stage to which the real EEG samples belonged. From multiple real EEG sample pairs, identify real EEG sample pairs that are adjacent to each sleep stage, and generate virtual EEG sample pairs based on the real EEG sample pairs that are adjacent to each sleep stage; the virtual EEG sample pairs include virtual EEG samples and virtual sample labels used to characterize the sleep stage to which the virtual EEG samples belong. The sleep staging model is trained based on virtual and real EEG sample pairs and is used to classify sleep stages.

[0106] The above method and steps, based on adjacent real EEG sample pairs in sleep stages, achieve data augmentation and obtain high-quality virtual EEG sample pairs that conform to physiological laws, which can be well adapted to the training scenario of sleep staging model. Model training based on virtual EEG sample pairs that conform to physiological laws can avoid interference with the decision boundary of sleep staging model caused by training with dirty data, and can improve the discrimination ability of sleep staging model in ambiguous states.

[0107] Bus 630 is used to connect different components of electronic device 600 and may include a data bus, an address bus and a control bus.

[0108] Electronic device 600 can communicate with one or more external devices 700 (such as keyboard, mouse, external controller, etc.) through I / O interface 640.

[0109] Electronic device 600 can communicate with one or more networks via network adapter 650. For example, network adapter 650 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication. Network adapter 650 can communicate with other modules of electronic device 600 via bus 630.

[0110] although Figure 6 As not shown in the diagram, other hardware and / or software modules may also be configured in the electronic device 600, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (Redundant Arrays of Independent Disks) systems, tape drives, and data backup storage systems.

[0111] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0112] Those skilled in the art will understand that various aspects of this disclosure can be implemented as systems, methods, or program products. Therefore, various aspects of this disclosure can be embodied in entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuit,” “module,” or “system.” Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0113] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is defined only by the appended claims.

Claims

1. A sleep staging processing method, characterized by, The method includes: Multiple pairs of real brainwave samples are acquired, wherein each pair of real brainwave samples includes real brainwave samples and real sample labels used to characterize the sleep stage to which the real brainwave samples belong. From the plurality of real EEG sample pairs, adjacent real EEG sample pairs of sleep stages are determined, and virtual EEG sample pairs are generated based on the adjacent real EEG sample pairs of sleep stages; the virtual EEG sample pairs include virtual EEG samples and virtual sample labels used to characterize the sleep stage to which the virtual EEG samples belong. The sleep staging model is trained based on the virtual EEG sample pairs and the real EEG sample pairs. The sleep staging model is used to classify sleep stages. The step of determining adjacent real brainwave sample pairs during sleep stages from the plurality of real brainwave sample pairs includes: Based on sleep stage transition constraints, pairs of real EEG samples adjacent to each other in sleep stages are determined from the plurality of real EEG sample pairs; wherein, the sleep stage transition constraints include the pre-configured transition capabilities between different sleep stages; The step of generating virtual EEG sample pairs based on adjacent real EEG sample pairs during the sleep stage includes: Obtain the interpolation weight coefficients; wherein, the interpolation weight coefficients are determined based on a preset probability distribution; Based on the interpolation weighting coefficient, the pairs of real EEG samples adjacent to each other during the sleep stage are linearly interpolated to obtain virtual EEG sample pairs.

2. The method of claim 1, wherein, The sleep staging model includes an encoder and a classifier. Training the sleep staging model based on the virtual EEG sample pairs and the real EEG sample pairs includes: The virtual brainwave samples in the virtual brainwave sample pair and the real brainwave samples in the real brainwave sample pair are respectively extracted and encoded by the encoder in the sleep staging model to obtain the feature representation of the virtual brainwave sample and the feature representation of the real brainwave sample. The feature representations of the virtual EEG samples and the feature representations of the real EEG samples are used to classify sleep stages through the classifier in the sleep staging model, respectively, to obtain the sleep stage prediction results of the virtual EEG samples and the sleep stage prediction results of the real EEG samples. The model parameters of the sleep staging model are updated based on the first difference and the second difference; wherein, the first difference is the difference between the sleep stage prediction result of the virtual EEG sample and the virtual sample label of the virtual EEG sample; and the second difference is the difference between the sleep stage prediction result of the real EEG sample and the real sample label of the real EEG sample.

3. The method of claim 2, wherein, The method further includes: Obtain a pre-trained target network; wherein the target network is an encoder-decoder structure network used to reconstruct the time-frequency spectrum of electroencephalograms; Based on the parameters corresponding to the encoder in the target network, the parameters corresponding to the encoder in the sleep staging model are determined.

4. The method of claim 3, wherein, The method further includes: Multiple unlabeled EEG samples were acquired and converted into time-frequency spectra to obtain the original time-frequency spectra corresponding to the multiple unlabeled EEG samples. The original time-frequency spectra corresponding to the multiple unlabeled EEG samples are masked in the time domain and / or frequency domain to obtain the incomplete time-frequency spectra corresponding to the multiple unlabeled EEG samples. The incomplete time-frequency spectra corresponding to the multiple unlabeled EEG samples are reconstructed through the target network to obtain the reconstructed time-frequency spectra corresponding to the multiple unlabeled EEG samples. The target network is trained based on the original time-frequency spectra and reconstructed time-frequency spectra corresponding to the multiple unlabeled EEG samples.

5. The method of claim 3, wherein, After determining the parameters corresponding to the encoder in the sleep staging model based on the parameters corresponding to the encoder in the target network, the method further includes: Based on the preset parameter freezing strategy, the freezing parameters are determined from the parameters corresponding to the encoder in the sleep staging model; The frozen parameters are controlled not to be updated during the training process of the sleep staging model.

6. A sleep staging processing apparatus characterized by comprising: The device includes: The real sample pair acquisition module is used to acquire multiple real EEG sample pairs, wherein the real EEG sample pair includes real EEG samples and real sample labels used to characterize the sleep stage to which the real EEG samples belong. The virtual sample pair generation module is used to determine adjacent real EEG sample pairs from the plurality of real EEG sample pairs, and generate virtual EEG sample pairs based on the adjacent real EEG sample pairs of sleep stages; the virtual EEG sample pair includes a virtual EEG sample and a virtual sample label used to characterize the sleep stage to which the virtual EEG sample belongs. The sleep staging model training module is used to train the sleep staging model based on the virtual EEG sample pairs and the real EEG sample pairs. The sleep staging model is used to classify sleep stages. The step of determining adjacent real brainwave sample pairs during sleep stages from the plurality of real brainwave sample pairs includes: Based on sleep stage transition constraints, pairs of real EEG samples adjacent to each other in sleep stages are determined from the plurality of real EEG sample pairs; wherein, the sleep stage transition constraints include the pre-configured transition capabilities between different sleep stages; The step of generating virtual EEG sample pairs based on adjacent real EEG sample pairs during the sleep stage includes: Obtain the interpolation weight coefficients; wherein, the interpolation weight coefficients are determined based on a preset probability distribution; Based on the interpolation weighting coefficient, the pairs of real EEG samples adjacent to each other during the sleep stage are linearly interpolated to obtain virtual EEG sample pairs.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 5.

8. An electronic device, comprising: include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 5 by executing the executable instructions.

Images