A sleep staging method and system based on reinforcement learning

Abstract

The application discloses a sleep staging method and system based on reinforcement learning, comprising the following steps: electroencephalogram acquisition; electroencephalogram signal preprocessing; constructing an action policy network for feature extraction; setting a reward function; updating the action policy network; and outputting a classification result. The application proposes a better sleep electroencephalogram signal feature optimization based on a filtering type feature extraction method in the field of sleep staging, which can eliminate redundant features while maintaining accuracy. The application focuses on the effective information provided by the original sleep electroencephalogram signal data and the associated information between features, obtains an optimal feature subset through continuous learning, considers the interaction between various sleep electroencephalogram signal features, and can dynamically adjust the feature combination to adapt to different data sets and task requirements, so as to remove redundant features in a sleep staging recognition model, and obtain a staging result through the sleep staging recognition model.

Description

Technical Field

[0001] This invention belongs to the field of digital health and data-driven medical technology, specifically providing a sleep staging method and system based on reinforcement learning. Background Technology

[0002] Sleep is a vital part of human physiological activity, crucial for maintaining physical health and cognitive function. We spend one-third of our lives asleep; quality sleep can enhance our immunity, boost metabolism, and improve memory, significantly impacting our quality of life. Sleep staging is the foundation of sleep quality assessment and plays a crucial role in various sleep studies. Understanding the different stages and phases of sleep is invaluable for disease diagnosis, treatment, and sleep quality evaluation.

[0003] Sleep staging is primarily divided into REM (Rapid Eye Movement) and non-REM (Non-Rapid Eye Movement) sleep stages. The American Academy of Sleep Medicine established sleep staging standards in 2007, classifying sleep into five categories: W (wakefulness), NREM (Near-Near-Earth Memory), and REM (Near-Earth Memory), with NREM further subdivided into N1, N2, and N3 stages. This new staging method is more realistic and has been widely used in sleep diagnosis and research. However, manual sleep staging requires specialized knowledge and is extremely time-consuming. Furthermore, human scoring is susceptible to bias. These disadvantages make manual sleep staging inefficient and expensive.

[0004] Sleep EEG data is characterized by its disordered and random nature, with numerous biological features based on linear or nonlinear dynamics serving as sleep stage characteristics. Currently, commonly used automatic sleep staging methods primarily rely on the analysis of EEG signals, determining sleep stages by detecting different frequency components in the EEG signal (including waveforms, adjacent short-term abrupt changes, or long-term periodic temporal features of multiple segments). These methods, which extract different EEG features as classification features for sleep stage identification, often suffer from feature redundancy, leading to problems such as overfitting during training or reduced recognition accuracy.

[0005] The early paper, doi:10.15888 / j.cnki.csa.006594, titled "Feature Selection Algorithm Based on Reinforcement Learning," proposed the idea of ​​using a table of expected values ​​for feature selection. Action output required consulting this continuously updated table. However, this method has drawbacks. When the state space becomes too large, maintaining the expected value table leads to reduced network model efficiency. Furthermore, this method can only output discrete actions, not continuous action sequences. In sleep staging based on filter-based feature extraction, the number of channels often increases, resulting in a larger one-dimensional feature space for state selection, which also fails to satisfy the requirement of outputting continuous feature selection behavior.

[0006] Therefore, this invention proposes a method for sleep EEG feature selection based on reinforcement learning, which completes the removal of redundant sleep EEG features and the selection of sleep EEG features by exploring and utilizing the direct output of continuous action sequences through reinforcement learning. Summary of the Invention

[0007] The purpose of this invention is to provide a method for removing redundant features of sleep EEG based on reinforcement learning and a sleep staging system, so as to obtain the optimal feature subset that can be used for classification and identification during sleep and to perform sleep staging identification.

[0008] An action policy network is used for feature selection. The action policy network outputs a subset of candidate features, which is then imported into a classification test network to obtain the corresponding stage accuracy. A baseline value is set, and the accuracy is used as the reward function. The policy gradient method from reinforcement learning is used to feed this back into the feature selection network, allowing for targeted training. After a certain number of rounds of exploration and utilization, the optimal feature subset for the current sleep stage is finally output.

[0009] This invention proposes a sleep staging method based on reinforcement learning. The specific steps of the technical solution are as follows: S1, a multi-channel EEG device collects scalp EEG signal data of the subject during sleep.

[0010] S2. Preprocess the collected EEG signal data to reduce artifact interference and obtain preprocessed EEG sleep data samples.

[0011] S3. Feature extraction is performed on the EEG signals, and then feature optimization is performed using an action policy network; specifically:

[0012] 3-1 Feature Extraction

[0013] The frequency-divided spectral density feature set of the preprocessed EEG signal is extracted using the Welch algorithm;

[0014] 3-2 Initial Screening

[0015] For the frequency-divided spectral density feature set, randomly select two frequency band features X and Y, and calculate based on the correlation coefficient ρ (X,Y) Perform a comparison, if ρ (X,Y) If the value of X or Y is greater than the set threshold ρ0, which is generally (0,1), then the frequency band feature X or Y is removed; otherwise, the two frequency band features X and Y are retained until the pairwise correlation coefficients of all frequency band features in the frequency density feature set after frequency division are compared, that is, the correlation coefficients are all less than or equal to the set threshold ρ0.

[0016]

[0017] Where, X i Y iLet X and Y represent the i-th element of frequency band feature X and the i-th element of frequency band feature Y, respectively. The mean of elements in X and Y, ρ (X,Y) The value of is between [–1, 1]. This value reflects the strength of the linear correlation between the two variables. The larger the absolute value, the stronger the correlation.

[0018] 3-3 Feature Optimization Using Action Policy Networks

[0019] The action policy network sets all features in the initial spectral density feature set as the state space. It then uses this network to select sleep EEG features from the initial spectral density feature set, obtaining a feature probability matrix indicating whether each feature should be added to the optimal feature subset. If the feature probability is greater than or equal to a probability threshold α, it is considered to be added to the optimal feature subset; otherwise, it is considered to be removed from the optimal feature subset. Finally, the staged candidate feature subset F is obtained. w ;

[0020] S4. Extract and evaluate feature subsets;

[0021] The phased candidate feature subset F w Input the data into the first classifier for sleep stage identification and obtain the corresponding accuracy acc; S5, reward function setting;

[0022] Set an appropriate reward function and use the exploration and exploitation of reinforcement learning to update the network parameters of the above action policy network;

[0023] Use the classification accuracy acc as a parameter to set the reward function R:

[0024]

[0025] in Representing network parameters The reward function is R; β is the scaling factor; acc t As the baseline value;

[0026]

[0027] in The parameters of the action policy network are Given the probability of making an action decision at a given time, calculate... After taking the logarithm, multiplying it by the corresponding R, summing the results, and then differentiating, we get... The network parameters of the next round of action policy network are updated using formula (4), specifically:

[0028]

[0029] in Here, ρ represents the network parameters for the t-th iteration;

[0030] S6: Obtain the final classification result based on the trained action decision model:

[0031] After the action policy network model converges, the output yields the optimal feature subset F. best The data is then input into a second classifier for sleep stage identification to obtain the final classification result.

[0032] Preferably, the preprocessing of the EEG signal data in step S2 includes smoothing filter, average reference removal, frequency division filtering, and signal segmentation.

[0033] Preferably, the frequency bands of the spectral density feature set in step 3-1 are: delta (0.5-4Hz), theta (4-8Hz), alpha (8-13Hz), sigma (11-16Hz), beta1 (13-22Hz), beta2 (22-30Hz), and gamma (30-40Hz).

[0034] Preferably, the action policy network employs a CNN composed of fully connected layers; wherein the input layer has N neurons and the output layer has 1 neuron; the input is the spectral density feature set F after initial screening. 1～N F 1～N ={F1,F2…F N The output is the feature F. j The feature probability matrix of whether to include the optimal feature subset, j = 1, 2, ..., N;

[0035] A second objective of this invention is to provide a sleep staging system based on reinforcement learning, comprising:

[0036] The EEG acquisition module collects scalp EEG signal data from the subject during sleep.

[0037] The EEG signal preprocessing module preprocesses the collected EEG signal data;

[0038] The sleep feature extraction module uses a convergent action strategy network model to optimize the features of EEG signals.

[0039] The classification and recognition module uses a second classifier to identify sleep stages.

[0040] A third objective of this invention is to provide a machine-readable storage medium, characterized in that it stores computer-executable instructions that, when executed, cause the computer to perform the reinforcement learning-based sleep staging method.

[0041] A fourth objective of this invention is to provide a computing device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the reinforcement learning-based sleep staging method.

[0042] The beneficial effects of this invention are:

[0043] By using the feature selection method proposed in this invention, the feature dimension can be reduced in the filter-based sleep EEG feature selection method, the overfitting and recognition accuracy reduction caused by redundant features can be optimized, and continuous feature selection results can be output.

[0044] This invention proposes a filtering-based feature extraction method for sleep staging, which can better optimize sleep EEG signal features, eliminating redundant features while maintaining accuracy. Unlike previous manual feature selection for sleep staging analysis, this invention focuses on the effective information provided by the raw sleep EEG data and the correlation information between features. Through continuous learning, an optimal feature subset is obtained. This not only considers the interaction between various sleep EEG signal features but also dynamically adjusts the feature combination to adapt to different datasets and task requirements, thereby removing redundant features in the sleep staging identification model and obtaining the staging result through this model. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of the present invention;

[0046] Figure 2 This is a schematic diagram illustrating the action selection process using the action strategy network proposed in this invention.

[0047] Figure 3 This is a diagram of the electroencephalogram (EEG) recording channel for a specific embodiment of the present invention. Detailed Implementation

[0048] The present invention will be further analyzed below with reference to specific embodiments.

[0049] As attached Figure 2 As shown, this invention proposes a sleep staging method based on reinforcement learning: First, the required EEG signals are acquired, and the EEG data undergoes preprocessing such as smoothing filtering, removal of average reference, and frequency division filtering. The processed EEG data is then segmented into 30-second segments. The spectral density features of the processed EEG data are calculated and optimized, and the resulting feature set is input into an action policy network for classification model training. Simultaneously, the obtained recognition results are used as a reward function to update the policy network, ultimately obtaining a complete sleep staging recognition result.

[0050] Step S1: Acquire scalp EEG signals during sleep in healthy subjects using a multi-channel EEG acquisition device. This embodiment uses a 12-electrode Neuroscan device to acquire EEG data, with a sampling frequency of 512Hz. The electrode caps are placed using the international 10-20 system electrode placement method. The 12 electrodes are Fp1, Fp2, F3, F4, F7, F8, C3, C4, T7, T8, P3, and P4. The reference electrode is placed on the right earlobe. The EEG channel positions are as follows: Figure 3 As shown.

[0051] Step S2: Preprocess the collected EEG data to reduce interference from artifacts; specifically:

[0052] (1) Smoothing filter: The classic mean smoothing filter is adopted. After testing, it was found that the mean value of the amplitude at the 95th percentile of the sleep EEG signal is about 30μV. Therefore, 30μV is used as the threshold. For signal points with an absolute value greater than the threshold, the mean value of the signal 5 seconds before and after the current signal point is used instead.

[0053] (2) Remove average reference: Calculate the average value of the 12 channels of EEG data after filtering, and subtract the average value of all channels from the signal value of each channel, thereby reducing the impact of single-point faults.

[0054] (3) Frequency division filtering: This embodiment adopts the classic frequency band division method. Considering that the calculation of brain connectivity generally requires "narrowband signals", the beta frequency band is divided into high beta and low beta, represented by beta1 and beta2 respectively. Considering that sleep stages may be related to sleep spindles, and the sleep in which sleep spindles appear is also called sigma sleep, the frequency bands finally divided by the FIR filter are: delta (0.5-4Hz), theta (4-8Hz), alpha (8-13Hz), beta1 (13-22Hz), beta2 (22-30Hz), gamma (30-40Hz), sigma (11-16Hz).

[0055] (4) epoch segmentation: data is segmented with 30s as the segment length to represent the sleep stage.

[0056] Step S3: Construct a reinforcement learning policy network for feature extraction. The specific calculation is as follows:

[0057] Using the data obtained in step S2 as input, a feature space is constructed by creating candidate EEG feature sets through analysis of relative power in a specific frequency band. The Welch method is used to calculate the power spectral density of the original data at sampling frequency f, denoted as the power spectral density matrix. Specifically, the signal is divided into K segments, a Fourier transform is performed on each segment, and finally, the power spectral density estimate P for each segment is calculated. k(f); for P k The power spectrum estimate P(f) is obtained by averaging the values ​​of (f).

[0058]

[0059] After obtaining the power spectral density estimate p(f), Boolean indexing is performed. The average value of the filtered spectrum data is calculated on the last dimension, and the feature set is obtained according to the frequency band and channel division.

[0060] For the frequency-divided spectral density feature set, randomly select two frequency band features X and Y, and calculate based on the correlation coefficient ρ (X,Y) Perform a comparison, if ρ (X,Y) If the value of X or Y is greater than the set threshold ρ0, which is generally (0,1), then the frequency band feature X or Y is removed; otherwise, the two frequency band features X and Y are retained until the pairwise correlation coefficients of all frequency band features in the frequency density feature set after frequency division are compared, that is, the correlation coefficients are all less than or equal to the set threshold ρ0.

[0061]

[0062] Where, X i Y i Let X and Y represent the i-th element of frequency band feature X and the i-th element of frequency band feature Y, respectively. The mean of elements in X and Y, ρ (X,Y) The value of is between [–1, 1]. This value reflects the strength of the linear correlation between the two variables. The larger the absolute value, the stronger the correlation.

[0063] Step S4: Set up the action policy network

[0064] The action policy network sets all features in the initial spectral density feature set as the state space, and then performs optimal feature selection through the action policy network. This invention designs an action policy network based on a CNN, consisting of fully connected layers, for sleep EEG feature selection, where the input layer has N neurons and the output layer has 1 neuron. The network input is the feature encoding F from the feature space set extracted from the feature extraction. 1～N F 1～N ={F1,F2…F N The output is a probability matrix indicating whether a single feature within the feature set should be added to the optimal feature subset. An α is set as a probability threshold: if the probability of a feature is greater than or equal to α, it is considered to be added to the optimal feature subset; if the probability of a feature is less than α, it is considered to be removed from the optimal feature subset.

[0065] In this invention, the probability threshold α is set to 0.5, and the optimal feature subset F for each stage is obtained through an action policy network. w .

[0066] The action policy network employs a fully connected neural network structure from deep learning, with 12 hidden layers, each fully connected, to fully learn and extract the complex relationships between features. As the number of layers increases, the number of neurons increases or decreases by 2N, 4N, 8N, 16N, 32N, 64N, 32N, 16N, 8N, 4N, and N, respectively. This configuration allows for multi-level abstract representation of features at different layers, thereby better capturing the correlation information between features.

[0067] Finally, through the softmax activation function, the model outputs the selection probability values ​​of the features. This means that each feature receives a corresponding probability, representing the likelihood that the feature should be selected into the optimal feature subset. The output of this one-dimensional matrix provides clear decision guidance for the action policy network in feature selection.

[0068] This invention employs a random forest as the first classifier, which consists of multiple decision trees. Each decision tree is a tree-based classifier that classifies data through a series of decision nodes and leaf nodes. In this invention, we selected 200 decision trees to construct the random forest model. This large number of decision trees helps improve the model's robustness and generalization ability, thus better adapting to different data characteristics and distributions.

[0069] When building a random forest model, a subset of candidate features is used to fit the model. This helps reduce the risk of overfitting and improves the model's generalization ability. Subsequently, we use a test set for prediction and compare the prediction results with the actual labels to calculate the accuracy. This series of steps constitutes a comprehensive evaluation of the random forest model's performance and provides a reliable reference for the model's effectiveness in practical applications.

[0070] Step S5: Set the reward function. Setting an appropriate reward function allows for the use of reinforcement learning's exploration and exploitation to update the network parameters of the action policy network described above.

[0071] Use the classification accuracy acc as a parameter to set the reward function R:

[0072]

[0073] in Representing network parameters The reward function is R; β is the scaling factor; acc t As the baseline value;

[0074]

[0075] in The parameters of the action policy network are Given the probability of making an action decision at a given time, calculate... After taking the logarithm, multiplying it by the corresponding R, summing the results, and then differentiating, we get...

[0076] The network parameters of the next round of action policy network are updated using formula (4), specifically:

[0077]

[0078] in Here, ρ represents the network parameters for the t-th iteration;

[0079] The relatively optimal feature subset F can be obtained through the action policy network. best , as a feature for sleep stage identification.

[0080] Step S6: Based on the trained action policy network, use a second classifier to identify sleep stages of EEG signals.

[0081] The second classifier uses ResNet as the system classifier. ResNet addresses the vanishing and exploding gradient problems that occur in traditional deep neural network structures as the number of network layers increases by introducing the concept of residual learning. This allows the network to be trained more deeply, thereby improving the model's generalization ability. The residual learning mechanism makes it easier for the network to learn identity mappings, reduces the complexity of feature representation, and improves the model's generalization and expressive power.

[0082] The core component of ResNet is the residual block, which consists of the input, two convolutional layers, and residual connections. Residual connections allow the input to bypass one or more layers directly and be added to the output, thus passing the original input information to deeper layers. This design allows the network to learn residual mappings (i.e., the difference between the original output and the desired output) rather than directly learning the entire mapping, making it easier to optimize and train.

[0083] The optimal feature subset is used as the feature input to ResNet for classification, and finally the sleep staging system is completed.

[0084] Reinforcement learning model training requires continuous iterative exploration and utilization. In this invention, training is required until the action policy network tends to converge, at which point the final feature selection method can be obtained. Only the method with the highest probability is output, that is, only the optimal feature subset is output. This optimal feature subset is then imported into the classifier, thus obtaining the final classification model, from which the final sleep staging result can be obtained.

[0085] To demonstrate the effectiveness of the proposed method, comparative experiments were conducted on the CAP sleep dataset: Method Origin performed sleep staging identification without redundant feature removal, while Method 2, based on the policy gradient-based reinforcement learning method of this invention, identified sleep stages. It can be seen that the proposed method maintains similar accuracy while reducing the number of features.

[0086] Table 1

[0087]

[0088] This invention provides an electronic device, specifically, the electronic device includes a memory and a processor, the memory stores executable code, and when the processor executes the executable code, it implements the method described in any of the embodiments.

[0089] The memory may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk drive. Communication between this system network element and at least one other network element is achieved through at least one communication interface (wired or wireless), which can use the Internet, wide area network, local area network, metropolitan area network, etc.

[0090] The bus can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc.

[0091] The memory is used to store programs. After receiving an execution instruction, the processor executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor or implemented by the processor.

[0092] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above methods can be completed by integrated logic circuits in the processor's hardware or by software instructions. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0093] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0094] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A sleep staging method based on reinforcement learning, characterized in that, The method includes the following steps: S1. Multi-channel EEG equipment collects scalp EEG signal data of subjects during sleep; S2. Preprocess the collected EEG signal data to reduce artifact interference and obtain preprocessed EEG sleep data samples. S3. Extract features from EEG signals and then optimize the features using an action policy network; Specifically: 3-1 Feature Extraction The frequency-divided spectral density feature set of the preprocessed EEG signal is extracted using the Welch algorithm; 3-2 Initial Screening Take two frequency band features from the frequency-divided spectral density feature set. , Based on the correlation coefficient Perform a comparison, if Greater than the set threshold If the frequency band feature X or Y is not found, then either frequency band feature X or Y is removed; otherwise, both frequency band features X and Y are retained. This process continues until the pairwise correlation coefficients of all frequency band features in the frequency-divided spectral density feature set are compared, meaning that all correlation coefficients are less than or equal to a set threshold. ; in, , Representing frequency band characteristics The i-th element, frequency band features The i-th element, , The mean values ​​of elements X and Y are respectively. 3-3 Feature Optimization Using Action Policy Networks The action policy network sets all features in the initial screening spectral density feature set as the state space, and uses the action policy network to select sleep EEG features from the initial screening spectral density feature set to obtain the feature probability matrix of whether each feature is added to the optimal feature subset. If the feature probability is greater than or equal to the probability threshold α, then it is considered to be added to the optimal feature subset; If the feature probability is less than α, then the optimal feature subset is considered to have been removed, and the final candidate feature subset for staging is obtained. ; S4. Extract and evaluate feature subsets; Phased candidate feature subsets Input the data into the first classifier for sleep stage identification and obtain the corresponding accuracy acc. S5. Reward function settings; Set a reward function and use the exploration and exploitation of reinforcement learning to update the network parameters of the above action policy network; Use the classification accuracy acc as a parameter to set the reward function R: in Representing network parameters The reward function R is as follows; It is a scaling factor; As the baseline value; in The parameters of the action policy network are The probability of making an action decision at a given time, first for Take the logarithm, and then use that logarithm with respect to the network parameters. Find the differential, multiply it by the corresponding reward R, and finally sum over all possible actions a to obtain the result. ; The network parameters of the next round of action policy network are updated using formula (4), specifically: in Let be the network parameters for the t-th iteration. Indicates the learning rate; S6. Obtain the final classification result based on the trained action decision model: After the action policy network model converges, its output yields the optimal feature subset. The data is then input into a second classifier for sleep stage identification to obtain the final classification result.

2. The method according to claim 1, characterized in that... In step S1, the sampling frequency of the EEG signal acquisition is 512Hz, and the positions of the acquisition electrodes are Fp1, Fp2, F3, F4, F7, F8, C3, C4, T7, T8, P3, and P4; the reference electrode is placed on the right earlobe.

3. The method according to claim 1, characterized in that... The preprocessing of EEG signal data in step S2 includes smoothing filter, average reference removal, frequency division filtering, and signal segmentation.

4. The method according to claim 1, characterized in that, The frequency bands of the spectral density feature set in step 3-1 are: delta, theta, alpha, sigma, beta1, beta2, and gamma bands.

5. The method according to claim 1, characterized in that, The action policy network adopts a CNN composed of fully connected layers; wherein the input layer has N neurons and the output layer has 1 neuron; The input is the spectral density feature set after initial screening. , ={ , … The output consists of various features. The feature probability matrix of whether to include the optimal feature subset, j=1,2,…,N.

6. The method according to claim 1, characterized in that... The probability threshold α is set to 0.

5.

7. The method according to claim 1, characterized in that... The first classifier uses the random forest algorithm; the second classifier uses the ResNet classifier.

8. A sleep staging system for implementing the method of any one of claims 1-7, characterized in that... include: The EEG acquisition module collects scalp EEG signal data from the subject during sleep. The EEG signal preprocessing module preprocesses the collected EEG signal data; The sleep feature extraction module uses a convergent action strategy network model to optimize the features of EEG signals. The classification and recognition module uses a second classifier to identify sleep stages.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-7.

10. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-7.

Images