Sleep data staging method and sleep data staging equipment

Through the sleep data staging model of multi-scale convolutional network, attention mechanism and bidirectional gating cyclic unit, combined with EEG and EOEG data, the problem of insufficient use of data modes and insufficient attention to characteristics is solved, and the problem of insufficient use of data modes is alleviated. The problem of data imbalance significantly improves the accuracy of sleep data staging.

CN118981664APending Publication Date: 2024-11-19CENT FOR EXCELLENCE IN BRAIN SCI & INTELLIGENCE TECH CHINESE ACAD OF SCI
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411070457.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

There are problems in the existing automatic sleep staging methods that are insufficiently used in data modes, insufficient feature attention and data imbalance, resulting in poor classification results, especially for the staging accuracy of N1 and REM periods.

Method used

A sleep data staging model composed of a multi-scale convolutional network, attention mechanism module and bidirectional gating cyclic unit is used to extract multimodal features with EEG and EOEG data, and the FE-Smote data enhancement method and weight Cross-entropy loss function alleviates data imbalance problem.

Benefits of technology

The robustness and feature extraction ability of the sleep data staging model were improved, and the classification accuracy of the sleep stage was enhanced, especially the staging effect of the N1 and REM stages was significantly improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118981664A_ABST
    Figure CN118981664A_ABST
Patent Text Reader

Abstract

The invention discloses a sleep data staging method and sleep data staging device.The sleep data staging method comprises the steps that sleep data to be staging are obtained, and the sleep data to be staging comprise electroencephalogram data and electrooculogram data; the sleep data to be staging is input into a pre-trained target sleep data staging model, a sleep data staging result output by the target sleep data staging model is obtained, and the target sleep data staging model is composed of a multi-scale convolutional network, an attention mechanism module, a bidirectional gating circulation unit and a classifier. According to the technical scheme, the technical problem that in an existing automatic sleep staging method, data modes are not fully used is solved, the robustness and the feature extraction capacity of the target sleep data staging model are improved, the influence of data imbalance is relieved, and therefore an accurate sleep data staging result is obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram monitoring and sleep analysis, and in particular to a sleep data staging method and a sleep data staging device. Background Art

[0002] In the early days, the American Academy of Sleep Medicine's Sleep and Related Events Scoring Manual divided sleep into: wakefulness, rapid eye movement (REM) and non-rapid eye movement (NREM), and the non-rapid eye movement stage was further divided into S1, S2, S3 and S4. Later, S3 and S4 were merged into N3, and N1, N2, and N3 were used as subdivisions of NREM. However, these processes are quite complicated and difficult to distinguish. In addition, according to the experience of medical experts, distinguishing between different stages of sleep requires a lot of manpower and time costs and is prone to human errors.

[0003] In order to save manpower, improve the accuracy of sleep stage classification and the applicability of wearable hardware, the research on automatic sleep staging based on single-channel EEG has begun to develop. In the early days, artificial features were mainly used to extract the time domain, frequency domain and nonlinear features of EEG signals, and then combined with traditional machine learning algorithms to perform sleep staging. However, the results are not very satisfactory. The biggest disadvantages of these methods include long time consumption, differences between different data sets and different individuals, and different characteristics to which they are adapted, so they cannot be generalized.

[0004] Due to these shortcomings, deep learning algorithms are used to automatically extract features, mainly by using CNN to extract time-invariant features and using recurrent networks such as RNN to automatically learn the transition rules between different sleep stages to extract time domain, frequency domain and nonlinear features in EEG data.

[0005] However, there are some problems with deep learning algorithms: On the one hand, the prediction effect of using single-channel EEG signals is not ideal in this case, especially for N1 and REM stages, where single-channel EEG signals contain limited information. On the other hand, imbalanced sleep data will also affect the model, and the lack of attention to important features when using deep learning algorithms will also affect the accuracy of sleep data staging results.

[0006] Therefore, the automatic sleep classification method based on deep learning currently still has problems such as insufficient use of data modalities, insufficient feature attention, and data imbalance, and the above problems need to be solved urgently. Summary of the invention

[0007] The present invention provides a sleep data staging method and a sleep data staging device, so as to improve the robustness and feature extraction capability of a target sleep data staging model, reduce the impact of data imbalance, and obtain a more accurate sleep data staging result.

[0008] According to one aspect of the present invention, a method for staging sleep data is provided, comprising:

[0009] Acquiring sleep data to be staged, wherein the sleep data to be staged includes electroencephalogram data and electrooculogram data;

[0010] The sleep data to be staged is input into a pre-trained target sleep data stage model to obtain a sleep data stage result output by the target sleep data stage model, wherein the target sleep stage model is composed of a multi-scale convolutional network, an attention mechanism module, a bidirectional gated recurrent unit and a classifier.

[0011] According to another aspect of the present invention, there is provided a sleep data staging device, comprising:

[0012] A sleep data acquisition module, used to acquire sleep data to be staged, wherein the sleep data to be staged includes electroencephalogram data and electrooculogram data;

[0013] The sleep data staging module is used to input the sleep data to be staged into a pre-trained target sleep data staging model to obtain the sleep data staging result output by the target sleep data staging model, wherein the target sleep staging model is composed of a multi-scale convolutional network, an attention mechanism module, a bidirectional gated recurrent unit and a classifier.

[0014] According to another aspect of the present invention, a sleep data staging device is provided, the sleep data staging device comprising:

[0015] at least one processor; and,

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the sleep data staging method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the sleep data staging method described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention obtains sleep data to be staged, wherein the sleep data to be staged includes electroencephalogram data and electrooculogram data; inputs the sleep data to be staged into a pre-trained target sleep data stage model, and obtains the sleep data stage result output by the target sleep data stage model, wherein the target sleep stage model is composed of a multi-scale convolutional network, an attention mechanism module, a bidirectional gated recurrent unit and a classifier. The technical solution of the embodiment of the present invention selects electroencephalogram data and electrooculogram data as input data of the automatic sleep classification model, which can avoid the instability of the model classification performance caused by the noise of the electroencephalogram data or the lack of feature information, improve the robustness of the model, and solve the problem of insufficient use of data modality in the existing automatic sleep staging method; in the process of extracting multimodal features of sleep, through the multi-scale convolution module, the attention mechanism module and the bidirectional gated recurrent unit, not only the time domain, frequency domain and nonlinear features of the data can be extracted, but also the extracted time-invariant features are weighted, and the features important for sleep classification can get more attention, thereby improving the ability of the model to extract features, solving the problem of insufficient feature attention, and obtaining a more accurate sleep data stage result.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a sleep data staging method provided in Embodiment 1 of the present invention;

[0023] Figure 2 A network structure diagram of a target sleep data staging model provided in the first embodiment of the present invention;

[0024] Figure 3 A flowchart of a sleep data staging method provided in Embodiment 2 of the present invention;

[0025] Figure 4 A schematic diagram of sample generation based on FE-smote provided in the second embodiment of the present invention;

[0026] Figure 5 A multi-scale convolutional network structure diagram provided in Example 2 of the present invention;

[0027] Figure 6 The overall structure diagram of the attention mechanism provided in the second embodiment of the present invention;

[0028] Figure 7 A structural diagram of a CAM module provided in Embodiment 2 of the present invention;

[0029] Figure 8 A structural diagram of a SAM module provided in Embodiment 2 of the present invention;

[0030] Fig. 9 A GRU unit structure diagram provided for the second embodiment of the present invention;

[0031] Fig.10 The expanded diagram of the BiGRU provided in the second embodiment of the present invention;

[0032] Fig.11 A schematic diagram of the structure of a sleep data staging device provided in Embodiment 3 of the present invention;

[0033] Fig.12 This is a schematic diagram of the structure of a sleep data staging device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] In order to further clarify the technical effects of the present invention, before introducing specific embodiments, the problems existing in the prior art are further explained as follows:

[0037] When using deep learning algorithms to stage sleep data, the prediction effect of using single-channel EEG signals is not ideal, especially for N1 and REM stages, where single-channel EEG signals contain limited information. However, in actual applications, when experts judge the sleep stage, if they cannot judge from the electroencephalogram (EEG), they will also refer to the relevant information of the electrooculogram (EOG), and related studies have also shown that sleep is closely related to EEG, EMG and EOG signals. In other words, the existing technology has the problem of insufficient use of data modalities.

[0038] Moreover, the inventors believe that in the feature extraction process, features that are more relevant to the sleep stage should receive more attention. Therefore, it is necessary to explore the applicability of few-lead multimodal signals in sleep staging and the problem of weight allocation of different features in the feature extraction process. That is, the prior art has the problem of insufficient feature attention.

[0039] In addition, in order to alleviate the impact of data imbalance on the model, data augmentation methods are used. Data augmentation methods usually use oversampling (such as synthetic minority oversampling technology) to balance samples by synthesizing minority categories, but random noise will be added during the synthesis process to affect the classification results of the model. The added samples in the sequence model will change the transition rules of the sleep stage, thereby changing the long-term temporal context information. In other words, the existing technology has a data imbalance problem.

[0040] In order to solve the problems of insufficient use of data modality, insufficient feature attention and data imbalance in existing sleep staging methods. The present invention proposes a hybrid CNN-Attention-BIGRU (convolutional neural network-self-attention mechanism-bidirectional gated recurrent unit) automatic sleep classification model using single-channel EEG-EOG and a data enhancement method based on feature engineering-smote (FE-Smote). The time-invariant features of EEG-EOG data are extracted by forming a feature extraction module with a multi-scale CNN layer and a convolutional block attention module (CABM). After the features are fused, the weights are assigned to the features according to the importance of the features and the sleep stage through the channel and spatial attention mechanism; the bidirectional gated recurrent unit (Bi-GRU) is used to capture the nonlinear dependencies in the time series and learn the conversion rules between different sleep stages, so as to realize the feature extraction of multimodal data and the accurate classification of sleep stages. In the above process, the FE-Smote data enhancement method and weighted cross entropy are used as LOSS (loss function) to alleviate the data imbalance problem.

[0041] Embodiment 1

[0042] Figure 1 This is a flowchart of a sleep data staging method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of accurately staging sleep data. The method can be executed by a sleep data staging device, which can be implemented in the form of hardware and / or software, and can be configured in a sleep staging device. Figure 1 As shown, the method includes:

[0043] S110, obtaining sleep data to be divided into stages.

[0044] The sleep data to be staged may be sleep data that needs to be staged, and the sleep data to be staged includes electroencephalogram data and electrooculogram data.

[0045] Specifically, the sleep data to be staged can be obtained through an external device that monitors sleep data. EEG data and electrooculogram data need to be obtained during the acquisition. Data based on multiple modalities can help accurately stage sleep. Selecting EEG and EOG as input data for the target sleep data staging model can avoid instability in the model classification performance due to EEG noise or lack of feature information.

[0046] S120, inputting the sleep data to be staged into a pre-trained target sleep data stage model to obtain a sleep data stage result output by the target sleep data stage model.

[0047] Among them, the target sleep staging model consists of a multi-scale convolutional network, an attention mechanism module, a bidirectional gated recurrent unit and a classifier.

[0048] Specifically, in the process of extracting multimodal features of sleep, due to the increase in data information, the feature extraction capability is required to be improved. Through the multi-scale convolution module, the attention mechanism module and the bidirectional gated recurrent unit, not only can the time domain, frequency domain and nonlinear features of the sleep data to be staged be extracted, but also the extracted time-invariant features are weighted, and the features important for sleep classification can get more attention. This can solve the problem of insufficient attention to features in the prior art and improve the model's ability to extract features to obtain more accurate sleep staging results.

[0049] Figure 2 This is a network structure diagram of the target sleep data staging model provided in Example 1 of the present invention.

[0050] In the embodiment of the present invention, the time-invariant features of EEG-EOG data are mainly extracted through a multi-scale CNN layer and feature fusion is performed; then, different weights are assigned to the fused features through the attention mechanism CBAM module, so that the features important for sleep classification receive more attention; Bi-GRU is selected to capture the nonlinear dependencies in the time series and learn the transition rules between different sleep stages. Finally, a fully connected layer and a Softmax layer are used to form a classifier. The fully connected layer effectively fuses the information between the features through dimensionality reduction decoding to obtain the vector corresponding to the label. Finally, the probability distribution of each label is obtained through the Softmax function, and the label with the highest probability is used as the prediction of the sleep stage of the model at this time.

[0051] Based on the above technical solution, the classifier consists of a fully connected layer and a Softmax layer. The fully connected layer is similar to a linear combination of a multi-layer perceptron, which further reduces the dimensionality of multiple input features and plays an important role in improving the model's expressiveness, reducing redundant feature maps, and reducing the computational burden in deep networks. As the last layer of the entire model, the Softmax layer obtains the feature results of each category to calculate the probability of belonging to each category.

[0052] The technical solution of the embodiment of the present invention obtains sleep data to be staged, wherein the sleep data to be staged includes electroencephalogram data and electrooculogram data; inputs the sleep data to be staged into a pre-trained target sleep data stage model, and obtains the sleep data stage result output by the target sleep data stage model, wherein the target sleep stage model is composed of a multi-scale convolutional network, an attention mechanism module, a bidirectional gated recurrent unit and a classifier. The technical solution of the embodiment of the present invention selects electroencephalogram data and electrooculogram data as input data of the automatic sleep classification model, which can avoid the instability of the model classification performance caused by the noise of the electroencephalogram data or the lack of feature information, improve the robustness of the model, and solve the problem of insufficient use of data modality in the existing automatic sleep staging method; in the process of extracting multimodal features of sleep, through the multi-scale convolution module, the attention mechanism module and the bidirectional gated recurrent unit, not only the time domain, frequency domain and nonlinear features of the data can be extracted, but also the extracted time-invariant features are weighted, and the features important for sleep classification can get more attention, thereby improving the ability of the model to extract features, solving the problem of insufficient feature attention, and obtaining a more accurate sleep data stage result.

[0053] Embodiment 2

[0054] Figure 3 This is a flowchart of a sleep data staging method provided in Example 2 of the present invention. This example further introduces in detail the training process of the target sleep data staging model based on the above example. The specific implementation method can refer to the technical solution of this example. Among them, the technical terms that are the same or corresponding to the above example are not repeated here. Figure 3 As shown, the method includes:

[0055] S210 , obtaining sample sleep data, dividing the sample sleep data into a plurality of sleep data segments, taking the sleep data segments and corresponding stage labels as original samples, and generating an original sample set based on the plurality of original samples.

[0056] Among them, the sample sleep data can be pre-acquired sleep data, such as EEG and EOG; the stage label can be a label corresponding to the sleep data segment, the stage label is used to indicate the sleep stage of the sleep data segment, each sleep data segment corresponds to a stage label, the sleep data segment and the stage label of the sleep data segment constitute an original sample, and multiple original samples constitute an original sample set.

[0057] Specifically, the EEG and EOG data of the subjects during sleep can be obtained, and then the data is labeled with sleep stages, and the sleep stages are divided into wakefulness (W), rapid eye movement (REM) and non-rapid eye movement (N1, N2, N3). The data is cut into 30-second data segments, that is, cut into multiple sleep data segments. After the cutting is completed, each sleep data segment is aligned with the corresponding label as the original sample, and multiple original samples constitute the original sample set.

[0058] S220 , processing the original sample set by using a synthetic minority oversampling algorithm to obtain a newly added sample set, and screening the newly added samples in the newly added sample set by using a target voter to obtain a target newly added sample set.

[0059] It should be noted that since the number of samples corresponding to some sleep stages in the original sample set is relatively small, there is a problem of sample imbalance. Therefore, the synthetic minority oversampling technique (SMOTE) can be used to process the original sample set. This algorithm increases the number of minority class samples by synthesizing new minority class samples, thereby achieving the purpose of balancing the data set. The newly added samples can constitute a newly added sample set. The target voter can be understood as a voter composed of multiple decision models, which is used to select a part from the newly added sample set to constitute the target newly added sample set.

[0060] In an embodiment of the present invention, the newly added samples include newly added sleep data segments and newly added stage labels; the newly added samples in the newly added sample set are screened by the target voter to obtain the target newly added sample set, including: decomposing the sleep data segments of the original samples and the newly added sleep data segments of the newly added samples into waveforms of different frequency ranges by wavelet decomposition, and selecting the waveform of 0-40HZ for reconstruction to obtain reconstructed waveform data; extracting the time domain, frequency domain and nonlinear features of the reconstructed waveform data, selecting the average power, sample entropy, wavelet entropy and differential entropy features of the reconstructed waveform data, and obtaining the original sample feature data corresponding to the original sample and the newly added sample feature data corresponding to the newly added sample; using the original sample feature data and the stage labels corresponding to the original sample feature data as voter training samples, and training the initial voter based on the voter training samples to obtain the target voter; inputting the newly added sample feature data into the target voter, and when the voting result output by the target voter is consistent with the newly added stage label, using the newly added sample as the target newly added sample; and determining the target newly added sample set based on the target newly added sample.

[0061] The initial voter is composed of at least two classifiers among K-Nearest Neighbors (KNN), Support Vector Machine (SVM) and Random Forest. The initial voter can be an untrained voter. After the training of the initial voter is completed, the target voter is obtained.

[0062] Specifically, after wavelet decomposition and reconstruction processing of the sleep data segments of the original samples and the newly added sleep data segments of the newly added samples, the feature data related to the sleep stages can be extracted. Then, the feature data of the original samples and the corresponding stage labels are used as training samples, and the initial voter is trained based on these training samples to obtain a target voter that can accurately identify sleep stages. Furthermore, the newly added samples are screened based on the target voter. Specifically, the feature data of the newly added samples can be input into the target voter. If the voting result output by the voter is consistent with the stage label of the newly added sample, the newly added sample is considered to be reliable and is used as the target newly added sample. Finally, all the target newly added samples screened out based on the above process constitute the target newly added sample set.

[0063] In a preferred embodiment, a small number of sample members S can be randomly selected based on SMOTE, and the Euclidean distance of each minority class sample to its nearest neighbor is calculated. Then, according to the set sampling ratio, S randomly selects sample S from the K nearest neighbor samples. r Generate new minority class samples S', generally set K = 5. The calculation formula is as follows:

[0064] S'=S+rand*(S r -S) (1)

[0065] Here rand = [0,1].

[0066] Based on SMOTE, the present invention proposes FE-Smote. In the process of sleep classification, the N1 stage is easily misclassified into the W and N2 stages. FE-Smote generates a certain number of false samples through SMOTE, randomly samples the samples of W, N1, and N2 categories and combines them with the false samples of N1, and performs wavelet decomposition and reconstruction on the EEG data so that the reconstructed data has the characteristics of different frequency domains. Then, the time domain, frequency domain and nonlinear characteristics of the reconstructed data are extracted, and the average power, sample entropy, wavelet entropy and differential entropy characteristics of the data are selected.

[0067] The voting machine is composed of KNN, SVM and random forest classifiers. Randomly sample W, N1, N2 samples as training sets, and N1 fake samples as test sets. The classification results of KNN, SVM and random forest classifiers are all N1. The fake samples are regarded as real samples, so as to remove samples with a lot of noise. Figure 4 A schematic diagram of sample generation based on FE-smote provided in the second embodiment of the present invention.

[0068] S230: Training a first sleep staging model based on the original sample set and the target newly added sample set to obtain a sleep staging model to be used.

[0069] Among them, the first sleep staging model consists of a multi-scale convolutional network, an attention mechanism module and a classifier.

[0070] It is understandable that since the new samples are generated by smote, the new samples do not contain time information. The original sample set and the target new samples can be used as the training set of the first sleep staging model to train the first sleep staging model to obtain the staging model to be used.

[0071] In an embodiment of the present invention, the first sleep staging model is trained based on the original sample set and the target newly added sample set to obtain the sleep staging model to be used, including: taking a single sample in the original sample set or the target newly added sample set as a sample to be input, and inputting the sample to be input into the multi-scale convolutional network to obtain an initial feature vector; inputting the initial feature vector into the attention mechanism module to obtain a target feature map; inputting the target feature map into the classifier to obtain a first staging result, and based on the first staging result and the staging label of the sample to be input, correcting the first model parameters of the first sleep staging model to obtain the sleep staging model to be used.

[0072] Among them, the sample to be input can be understood as the sample that needs to be input into the multi-scale convolutional network, and the sample to be input can be a sample in the original sample set or the target newly added sample set; the initial feature vector can be understood as the feature vector obtained after the multi-scale convolutional network performs feature extraction on the input sample; the target feature map refers to the feature map obtained after the attention mechanism module processes the initial feature vector; the first staging result can be the prediction result obtained after the first sleep staging model predicts the sleep staging of the input sample; after the training of the first sleep staging model is completed, it becomes the sleep staging model to be used.

[0073] Specifically, a sample is selected from the original sample set or the target newly added sample set as the sample to be input, and the sample to be input is input into the multi-scale convolutional network. The multi-scale convolutional network processes the input sample, extracts features of different scales, and obtains an initial feature vector. Then, the initial feature vector is input into the attention mechanism module, and the attention mechanism module performs weighted processing on the feature vector to highlight important features and suppress unimportant features to obtain a target feature map. Further, the target feature map is input into the classifier, and the classifier judges the sleep stage according to the target feature map to obtain a first staging result. The first staging result is compared with the staging label of the sample to be input, and the loss function is calculated. Based on the loss function, the first model parameters of the first sleep staging model are corrected to optimize the performance of the model. Repeat the above steps until certain training conditions (such as the number of iterations, loss function threshold, etc.) are met. After the training is completed, the optimized sleep staging model to be used is obtained.

[0074] In an embodiment of the present invention, Figure 5 The multi-scale convolutional network structure diagram provided in the second embodiment of the present invention is as follows: Figure 5 As shown, the sleep data segment and the newly added sleep data segment both include an electroencephalogram data segment and an electrooculogram data segment; the multi-scale convolutional network includes a first network branch and a second network branch, and the first network branch and the second network branch both include three convolution blocks, two maximum pooling layers, and two random inactivation layers;

[0075] The step of inputting the sample to be input into the multi-scale convolutional network to obtain an initial feature vector includes: inputting an EEG data segment of the sample to be input into the first network branch to obtain a first EEG feature vector output by the first network branch; inputting an EEG data segment of the sample to be input into the second network branch to obtain a second EEG feature vector output by the second network branch;

[0076] Input the electrooculogram data segment of the sample to be input into the first network branch to obtain a first electrooculogram feature vector output by the first network branch; input the electrooculogram data segment of the sample to be input into the second network branch to obtain a second electrooculogram feature vector output by the second network branch;

[0077] The first EEG feature vector, the second EEG feature vector, the first electrooculogram feature vector, and the second electrooculogram feature vector are fused to obtain an initial feature vector of the sample to be input.

[0078] Specifically, the EEG data segments of the samples to be input are sequentially input into the first network branch and the second network branch, and after being processed by the two network branches, a first EEG feature vector output by the first network branch and a second EEG feature vector output by the second network branch are obtained.

[0079] The electrogram data segments of the sample to be input are sequentially input into the first network branch and the second network branch, and after being processed by the two network branches, a first electrooculogram feature vector output by the first network branch and a second electrooculogram feature vector output by the second network branch are obtained. Further, the first EEG feature vector, the second EEG feature vector, the first electrooculogram feature vector and the second electrooculogram feature vector are fused to obtain an initial feature vector of the sample to be input.

[0080] Specifically, EEG data and EOG data are used as input and put into a multi-scale convolutional network. The waveform features of EEG data and EOG data are extracted through the same multi-scale convolutional network. The multi-scale feature extraction module (multi-scale convolutional network) consists of two branches with different one-dimensional convolution kernel sizes, each branch consists of 3 convolution blocks (including convolution layers, regularization layers and activation functions), two maximum pooling layers and two random dropout layers. After the convolution calculation is completed in the first convolution layer of each branch, a max-pooling layer is added to reduce the calculation time by reducing the number of parameters. In order to avoid overfitting, dropout (random dropout layer) is also applied after two maxpoolings. Finally, the outputs of the two branches are combined after feature extraction through a series operation, and then the max-poo ling layer is used to reduce the number of parameters to obtain the feature vector F, which is also the initial feature vector.

[0081] In an embodiment of the present invention, the attention mechanism module includes a channel attention module and a spatial attention module; the initial feature vector is input into the attention mechanism module to obtain a target feature map, including: performing spatial information aggregation on the initial feature vector through the channel attention module to obtain a channel attention map; multiplying the channel attention map with the initial feature vector to obtain an intermediate feature map; processing the intermediate feature map through the spatial attention module to obtain a spatial attention map; multiplying the spatial attention map with the intermediate feature map to obtain the target feature map.

[0082] After the multi-scale convolutional network feature extraction, the attention mechanism Convolutional Block Attention Module (CBAM) is added. CBAM is divided into Channel Attention (CAM) and Spatial Attention (SAM) modules. The CAM module allows the network to focus on useful feature channels and ignore the rest. The SAM module allows the network to focus on local areas of interest in the feature map. After obtaining the multi-scale convolutional layer output feature vector F, the CAM module aggregates the spatial information of the feature map F through max-pooling and average-pooling (average pooling layer), generates two different types of features, max-pooled and average-pooled, and then generates the channel attention map M through the MLP network and the Sigmod function. c , M c The calculation formula is as follows:

[0083] M c =Sigmod(conv(MaxPool(F))+conv(AvgPool(F))) (2)

[0084] Get the channel attention map M c Multiply it with the feature vector F, and give the same weight to all elements of each channel in F to get the feature map F C (Intermediate feature map). Compared with F, it pays more attention to the channels that have a greater impact on the classification results. Get feature map F C After that, the maximum pooling and average pooling operations are performed through the SAM module, and the final spatial attention map MS is generated through the convolution layer after connection. The MS calculation formula is as follows:

[0085] M S Sigmod=(conv k [MaxPool(F C );(AvgPool(F C )]) (3)

[0086] Here k is the convolution kernel size, k = 7. Get the spatial attention M S After the figure, and then with the feature map F C Multiply them together and apply the same weight to the same position of each channel to obtain the feature map F CS , to focus on which position in the data is more effective. The CAM and SAM modules improve the adaptability of the neural network to variable-length input sequences through these operations. The whole process is shown in the following formula:

[0087] F c =Mc (F)*F (4)

[0088] F cs =M s (F c )*F c (5)

[0089] Here F∈R C×L , M c ∈R C×L , F c ∈R C×L , M S ∈R C×L , F CS ∈R C×L , * is the element-wise multiplication, Figure 6 The overall structure diagram of the attention mechanism provided in the second embodiment of the present invention is as follows: Figure 7 The structure diagram of the CAM module provided in the second embodiment of the present invention is as follows: Figure 8 This is a structural diagram of the SAM module provided in Example 2 of the present invention.

[0090] S240: construct a second sleep staging model based on the bidirectional gated recurrent unit and the sleep staging model to be used, and train the second sleep staging model based on the original training sample set to obtain the target sleep staging model.

[0091] The second sleep staging model is a model obtained by combining the sleep staging model to be used with the gated sequential unit. Fig. 9 The expanded diagram of BiGRU provided in the second embodiment of the present invention is: Fig.10 This is a GRU unit structure diagram provided for Embodiment 2 of the present invention.

[0092] In an embodiment of the present invention, the training of the second sleep staging model based on the original training sample set to obtain the target sleep staging model includes: connecting the input end of the bidirectional gated recurrent unit to the output segment of the attention mechanism module in the sleep staging model to be used, and connecting the output end of the bidirectional gated recurrent unit to the input end of the classifier to obtain the second sleep staging model; inputting the original samples in the original training sample set into the second sleep staging model to obtain the second staging result output by the second sleep staging model; and correcting the second model parameters of the second sleep staging model based on the second staging result and the staging label of the original sample to obtain the target sleep staging model.

[0093] Specifically, the input end of the bidirectional gated recurrent unit is connected to the output end of the attention mechanism module in the sleep staging model to be used. In this way, the important features extracted by the attention mechanism module can be used as the input of the bidirectional gated recurrent unit to further capture the dependencies on the time series. The output end of the bidirectional gated recurrent unit is connected to the input end of the classifier to form a complete second sleep staging model. Furthermore, the original samples in the original training sample set are input into the second sleep staging model, and the model processes the input original samples and outputs the second staging result. Then, the second staging result is compared with the staging label of the original sample, and the loss function is calculated. The second model parameters of the second sleep staging model are corrected based on the loss function to optimize the performance of the model. Repeat the above training process until certain training conditions (such as the number of iterations, loss function threshold, etc.) are met. After the training is completed, the optimized target sleep staging model is obtained.

[0094] Through this process, the bidirectional gated recurrent unit is used to capture the time series dependency in the sleep data, and the attention mechanism module and classifier are combined to achieve accurate judgment of sleep stages. Finally, through continuous iterative training and optimization of model parameters, a target sleep stage model with good performance is obtained.

[0095] In a preferred embodiment, F is obtained from CBAM. cs , and then the nonlinear dependencies in the time series are captured by BIGRU, and the transition rules between different sleep stages are learned. The GRU (Gated Recurrent Unit) model is a recurrent neural network (RNN) architecture. Compared with traditional RNN, GRU and LSTM can overcome the problem of long-term memory loss to a certain extent. The difference is that GRU inherits the advantages of RNN in calculating time series, and simplifies the gated unit of LSTM by integrating the forget gate and input gate into the update gate. GRU consists of a reset gate, an update gate, candidate hidden states and hidden states. The main formula of GRU is as follows:

[0096] R t =σ(X t W xr +H t-1 W hr +B r ) (6)

[0097] Z t =σ(X t W xz +H t-1 W hz +B z ) (7)

[0098]

[0099] y t ={σ(W} o H t +B o ) (10)

[0100] Here σ(·) represents the sigmoid activation function, (W xr ,W hr ,W xz ,W hz ,W xh ,W hh ,W o ) and (B r ,B z ,B h ,B o ) represent the set of weight matrix and bias vector respectively, tanh is the hyperbolic tangent function, ⊙ is the Hadarard product, H t-1 is the state of the previous moment. The reset gate is mainly used to consider the amount of past information retained, and the update gate is used to control which past information is retained to the next moment. The state of GRU is updated by the reset gate and the update gate. With H t , output y t Mainly by the current state H t BiGRU is an extension of GRU. It consists of two GRUs: a forward GRU to accept forward input and a reverse GRU to accept reverse input. The output is determined by the current state of the forward GRU and the reverse GRU to extract the bidirectional temporal relationship of features, thereby learning more features to improve the performance of the model. The main process is as follows:

[0101]

[0102] Here, W f ,W r is the weight matrix, B bi is the bias matrix.

[0103] In the embodiment of the present invention, preferably, the first model parameter or the second model parameter is corrected based on a weighted cross entropy function, and the weighted cross entropy function is expressed as follows:

[0104]

[0105] In which, the actual label of an input data is replaced by a y l (one-hot encoding), when dealing with classification problems, the actual category label of each sample is converted into a one-hot encoded vector.

[0106] One-hot encoding is a commonly used method to represent categorical variables. For a classification problem with M categories, each category can be represented as an M-dimensional vector, where the index position corresponding to the category is 1 and the rest are 0.

[0107] Where N is the number of samples, M is the number of categories, i is the label representing the model input data (i=1,2,…M), is the probability distribution of the model prediction output, and is the number of samples in each category.

[0108] S250, obtaining sleep data to be divided into stages.

[0109] S260: Input the sleep data to be staged into a pre-trained target sleep data stage model to obtain a sleep data stage result output by the target sleep data stage model.

[0110] In one embodiment of the present invention, the publicly accessible Sleep-edf and ISRUC databases can be selected as data sets, and the model proposed by the present invention is used for testing, using a 10-fold cross-validation method. The data is evenly divided into 10 parts by the subjects, 9 of which are used as training sets and the rest as test sets, and the optimal model results are obtained through multiple cross-validations. The test results are compared with the advanced methods, and the results are shown in Table 1. From Table 1, it can be seen that the method proposed by the present invention can achieve better performance than the advanced methods.

[0111] Table 1

[0112]

[0113]

[0114] The technical solution of the embodiment of the present invention selects EEG and EOG as input data of the automatic sleep classification model, which can avoid the instability of the model classification performance caused by EEG noise or lack of feature information, and improve the robustness of the model. In the process of multimodal feature extraction of sleep, due to the increase of data information, the feature extraction capability is required to be improved. Through the multi-scale CNN module, CBAM module and BIGRU module, not only the time domain, frequency domain and nonlinear features of the data can be extracted, but also weights are applied to the extracted time-invariant features, and the features important for sleep classification can get more attention, improving the model's ability to extract features. Through the FE-Smote method and the weighted cross entropy function, the number of samples with a small number of sleep categories is increased and the generated samples are filtered, which effectively improves the phenomenon of over-classification of sleep levels caused by data imbalance. The weighted cross entropy function can be used as a loss function to focus on the categories with a small number of sleep stages during network training, further improving the impact of data imbalance.

[0115] Embodiment 3

[0116] Fig.11 This is a schematic diagram of the structure of a sleep data staging device provided in Embodiment 3 of the present invention. Fig.11 As shown, the device comprises:

[0117] The sleep data acquisition module 310 is used to acquire the sleep data to be staged, wherein the sleep data to be staged includes electroencephalogram data and electrooculogram data;

[0118] The sleep data staging module 320 is used to input the sleep data to be staged into a pre-trained target sleep data staging model to obtain the sleep data staging result output by the target sleep data staging model, wherein the target sleep staging model is composed of a multi-scale convolutional network, an attention mechanism module, a bidirectional gated recurrent unit and a classifier.

[0119] The technical solution of the embodiment of the present invention selects EEG and EOG as input data of the automatic sleep classification model, which can avoid the instability of the model classification performance caused by EEG noise or lack of feature information, and improve the robustness of the model. In the process of multimodal feature extraction of sleep, due to the increase of data information, the feature extraction capability is required to be improved. Through the multi-scale CNN module, CBAM module and BIGRU module, not only the time domain, frequency domain and nonlinear features of the data can be extracted, but also weights are applied to the extracted time-invariant features, and the features important for sleep classification can get more attention, improving the model's ability to extract features. Through the FE-Smote method and the weighted cross entropy function, the number of samples with a small number of sleep categories is increased and the generated samples are filtered, which effectively improves the phenomenon of over-classification of sleep levels caused by data imbalance. The weighted cross entropy function can be used as a loss function to focus on the categories with a small number of sleep stages during network training, further improving the impact of data imbalance.

[0120] On the basis of the above technical solution, the sleep data staging device further includes: a target sleep data staging model training module 330, and the target sleep data staging model training module 330 includes:

[0121] An original sample generation submodule, used to obtain sample sleep data, divide the sample sleep data into a plurality of sleep data segments, and use the sleep data segments and corresponding stage labels as original samples, and generate an original sample set based on the plurality of original samples;

[0122] A new sample generation submodule is used to process the original sample set by using a synthetic minority oversampling algorithm to obtain a new sample set, and to screen the new samples in the new sample set by using a target voter to obtain a target new sample set;

[0123] A first sleep staging model training submodule, configured to train a first sleep staging model based on the original sample set and the target newly added sample set to obtain a sleep staging model to be used, wherein the first sleep staging model is composed of a multi-scale convolutional network, an attention mechanism module, and a classifier;

[0124] The second sleep staging model training submodule is used to construct a second sleep staging model based on a bidirectional gated recurrent unit and the sleep staging model to be used, and train the second sleep staging model based on the original training sample set to obtain the target sleep staging model.

[0125] On the basis of the above technical solution, the newly added samples include newly added sleep data segments and newly added stage labels; the newly added sample generation submodule includes:

[0126] A wavelet reconstruction and decomposition unit, used to decompose the sleep data segments of the original samples and the newly added sleep data segments of the newly added samples into waveforms of different frequency ranges by wavelet decomposition, and select a 0-40 Hz waveform for reconstruction to obtain reconstructed waveform data;

[0127] Extracting the time domain, frequency domain and nonlinear features of the reconstructed waveform data, selecting the average power, sample entropy, wavelet entropy and differential entropy features of the reconstructed waveform data, and obtaining the original sample feature data corresponding to the original sample and the newly added sample feature data corresponding to the newly added sample;

[0128] an initial voter training unit, configured to use the original sample feature data and the period labels corresponding to the original sample feature data as voter training samples, and train the initial voter based on the voter training samples to obtain a target voter, wherein the initial voter is composed of at least two classifiers among KNN, SVM and random forest;

[0129] a target newly added sample determination unit, configured to input the newly added sample feature data into the target voting device, and when the voting result output by the target voting device is consistent with the newly added installment label, take the newly added sample as the target newly added sample;

[0130] The target newly added sample set determining unit is configured to determine the target newly added sample set based on the target newly added samples.

[0131] Based on the above technical solution, the first sleep stage model training submodule includes:

[0132] an initial feature vector determination unit, configured to take a single sample in the original sample set or the target newly added sample set as a sample to be input, and input the sample to be input into the multi-scale convolutional network to obtain an initial feature vector;

[0133] A target feature map determining unit, used for inputting the initial feature vector into the attention mechanism module to obtain a target feature map;

[0134] The first sleep staging model training unit is used to input the target feature map into the classifier to obtain a first staging result, and based on the first staging result and the staging label of the sample to be input, correct the first model parameter of the first sleep staging model to obtain the sleep staging model to be used.

[0135] On the basis of the above technical solution, the sleep data segment and the newly added sleep data segment both include electroencephalogram data segments and electrooculogram data segments; the multi-scale convolutional network includes a first network branch and a second network branch, and the first network branch and the second network branch both include three convolution blocks, two maximum pooling layers and two random inactivation layers;

[0136] The initial feature vector determining unit comprises:

[0137] An EEG feature vector determination subunit, used for inputting the EEG data segment of the sample to be input into the first network branch to obtain a first EEG feature vector output by the first network branch;

[0138] Inputting the EEG data segment of the sample to be input into the second network branch to obtain a second EEG feature vector output by the second network branch;

[0139] an electrooculogram feature vector determining subunit, configured to input the electrooculogram data segment of the sample to be input into the first network branch, and obtain a first electrooculogram feature vector output by the first network branch;

[0140] Inputting the electrooculogram data segment of the sample to be input into the second network branch, and obtaining a second electrooculogram feature vector output by the second network branch;

[0141] The feature vector fusion subunit is used to fuse the first EEG feature vector, the second EEG feature vector, the first electrooculogram feature vector and the second electrooculogram feature vector to obtain the initial feature vector of the sample to be input.

[0142] Based on the above technical solution, the attention mechanism module includes a channel attention module and a spatial attention module;

[0143] The target feature map determining unit is specifically used to:

[0144] Through the channel attention module, spatial information aggregation is performed on the initial feature vector to obtain a channel attention map;

[0145] Multiplying the channel attention map by the initial feature vector to obtain an intermediate feature map;

[0146] Processing the intermediate feature map by the spatial attention module to obtain a spatial attention map;

[0147] The spatial attention map is multiplied by the intermediate feature map to obtain the target feature map.

[0148] On the basis of the above technical solution, the second sleep stage model training submodule is specifically used for:

[0149] Connecting the input end of the bidirectional gated recurrent unit to the output segment of the attention mechanism module in the sleep staging model to be used, and connecting the output end of the bidirectional gated recurrent unit to the input end of the classifier to obtain the second sleep staging model;

[0150] Inputting the original samples in the original training sample set into the second sleep staging model to obtain a second sleep staging result output by the second sleep staging model;

[0151] Based on the second staging result and the staging label of the original sample, the second model parameters of the second sleep staging model are modified to obtain the target sleep staging model.

[0152] On the basis of the above technical solution, the first model parameter or the second model parameter is corrected based on a weighted cross entropy function, and the weighted cross entropy function is expressed as follows:

[0153]

[0154] In which, the actual label of an input data is replaced by a y l , N is the number of samples, M is the number of categories, i is the label representing the model input data (i = 1, 2, ... M), p li is the probability distribution of the model prediction output, Num i is the number of samples in each category.

[0155] Based on the above technical solution, the classifier consists of a fully connected layer and a Softmax layer.

[0156] The sleep data staging device provided in the embodiment of the present invention can execute the sleep data staging method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0157] Embodiment 4

[0158] Fig.12A schematic diagram of the structure of a sleep data staging device provided for Embodiment 4 of the present invention. The sleep data staging device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The sleep data staging device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0159] like Fig.12 As shown, the sleep data staging device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the sleep data staging device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0160] Multiple components in the sleep data staging device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the sleep data staging device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0161] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a sleep data staging method.

[0162] In some embodiments, the sleep data staging method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the sleep data staging device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the sleep data staging method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the sleep data staging method in any other appropriate manner (e.g., by means of firmware).

[0163] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0165] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein may be implemented on a sleep data staging device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the sleep data staging device. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0167] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0168] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0169] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0170] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A sleep data staging method, characterized in that: include: Acquiring sleep data to be staged, wherein the sleep data to be staged includes electroencephalogram data and electrooculogram data; The sleep data to be staged is input into a pre-trained target sleep data stage model to obtain a sleep data stage result output by the target sleep data stage model, wherein the target sleep stage model is composed of a multi-scale convolutional network, an attention mechanism module, a bidirectional gated recurrent unit and a classifier.

2. The method according to claim 1, characterized in that The target sleep data staging model is trained in the following way: Acquire sample sleep data, divide the sample sleep data into a plurality of sleep data segments, use the sleep data segments and corresponding stage labels as original samples, and generate an original sample set based on the plurality of original samples; The original sample set is processed by a synthetic minority oversampling algorithm to obtain a newly added sample set, and the newly added samples in the newly added sample set are screened by a target voter to obtain a target newly added sample set; Based on the original sample set and the target newly added sample set, training a first sleep staging model to obtain a sleep staging model to be used, wherein the first sleep staging model is composed of a multi-scale convolutional network, an attention mechanism module, and a classifier; A second sleep staging model is constructed based on the bidirectional gated recurrent unit and the sleep staging model to be used, and the second sleep staging model is trained based on the original training sample set to obtain the target sleep staging model.

3. The method according to claim 2, characterized in that The newly added samples include newly added sleep data segments and newly added stage labels; the newly added samples in the newly added sample set are screened by the target voter to obtain a target newly added sample set, including: Decomposing the sleep data segments of the original samples and the newly added sleep data segments of the newly added samples into waveforms of different frequency ranges by wavelet decomposition, and selecting the waveform of 0-40 Hz for reconstruction to obtain reconstructed waveform data; Extracting the time domain, frequency domain and nonlinear features of the reconstructed waveform data, selecting the average power, sample entropy, wavelet entropy and differential entropy features of the reconstructed waveform data, and obtaining the original sample feature data corresponding to the original sample and the newly added sample feature data corresponding to the newly added sample; The original sample feature data and the phase labels corresponding to the original sample feature data are used as voter training samples, and an initial voter is trained based on the voter training samples to obtain a target voter, wherein the initial voter is composed of at least two classifiers among KNN, SVM and random forest; Inputting the feature data of the newly added sample into the target voting device, and when the voting result output by the target voting device is consistent with the newly added installment label, taking the newly added sample as the target newly added sample; The target new sample set is determined based on the target new sample.

4. The method according to claim 2, characterized in that: The step of training the first sleep staging model based on the original sample set and the target newly added sample set to obtain a sleep staging model to be used includes: Taking a single sample in the original sample set or the target newly added sample set as a sample to be input, and inputting the sample to be input into the multi-scale convolutional network to obtain an initial feature vector; Inputting the initial feature vector into the attention mechanism module to obtain a target feature map; The target feature map is input into the classifier to obtain a first staging result, and based on the first staging result and the staging label of the sample to be input, the first model parameter of the first sleep staging model is modified to obtain the sleep staging model to be used.

5. The method according to claim 4, characterized in that The sleep data segment and the newly added sleep data segment both include an electroencephalogram data segment and an electrooculogram data segment; the multi-scale convolutional network includes a first network branch and a second network branch, and the first network branch and the second network branch both include three convolution blocks, two maximum pooling layers, and two random inactivation layers; The step of inputting the sample to be input into the multi-scale convolutional network to obtain an initial feature vector includes: Inputting the EEG data segment of the sample to be input into the first network branch to obtain a first EEG feature vector output by the first network branch; Inputting the EEG data segment of the sample to be input into the second network branch to obtain a second EEG feature vector output by the second network branch; Inputting the electrooculogram data segment of the sample to be input into the first network branch to obtain a first electrooculogram feature vector output by the first network branch; Inputting the electrooculogram data segment of the sample to be input into the second network branch, and obtaining a second electrooculogram feature vector output by the second network branch; The first EEG feature vector, the second EEG feature vector, the first electrooculogram feature vector, and the second electrooculogram feature vector are fused to obtain an initial feature vector of the sample to be input.

6. The method according to claim 4, characterized in that The attention mechanism module includes a channel attention module and a spatial attention module; The step of inputting the initial feature vector into the attention mechanism module to obtain a target feature map includes: Through the channel attention module, spatial information aggregation is performed on the initial feature vector to obtain a channel attention map; Multiplying the channel attention map by the initial feature vector to obtain an intermediate feature map; Processing the intermediate feature map by the spatial attention module to obtain a spatial attention map; The spatial attention map is multiplied by the intermediate feature map to obtain the target feature map.

7. The method according to claim 2, characterized in that: The step of training the second sleep staging model based on the original training sample set to obtain the target sleep staging model includes: Connecting the input end of the bidirectional gated recurrent unit to the output segment of the attention mechanism module in the sleep staging model to be used, and connecting the output end of the bidirectional gated recurrent unit to the input end of the classifier to obtain the second sleep staging model; Inputting the original samples in the original training sample set into the second sleep staging model to obtain a second sleep staging result output by the second sleep staging model; Based on the second staging result and the staging label of the original sample, the second model parameters of the second sleep staging model are modified to obtain the target sleep staging model.

8. The method according to claim 4 or 7, characterized in that: The first model parameter or the second model parameter is modified based on a weighted cross entropy function, and the weighted cross entropy function is expressed as follows: In which, the actual label of an input data is replaced by a y l , N is the number of samples, M is the number of categories, i is the label representing the model input data (i = 1, 2, ... M), p li is the probability distribution of the model prediction output, Num i is the number of samples in each category.

9. The method according to claim 2, characterized in that: The classifier consists of a fully connected layer and a Softmax layer.

10. A sleep data staging device, characterized in that: The sleep data staging device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the sleep data staging method according to any one of claims 1 to 9.

Citation Information

Cited By

  • Intelligent detection method for electric vehicle charging equipment

    CN119780592A

  • An intelligent detection method for tram charging equipment

    CN119780592B

  • Personalized dynamic sleep staging method and system

    CN120267244A

  • Sleep stage identification method, device and equipment based on multi-mode signal and medium

    CN121337283A

  • A sleep staging system based on modeling of multi-physiological signal heterogeneous sequences

    CN122720932A