Electroencephalogram signal-based state detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a state detection method and device based on electroencephalogram signals, electronic equipment and a storage medium. The method comprises the following steps: in response to a signal acquisition request, determining a first signal acquired by an electroencephalogram electrode; performing spectrum feature extraction and spectrum feature splicing on the first signal to obtain a first image; and classifying the first signal through a first model according to the first image to obtain a first classification result. According to the method, the frequency spectrum feature extraction and the frequency spectrum feature splicing are performed on the first signal, so that the obtained first image can better describe the frequency feature change of the first signal, the sleep type to which the first signal belongs is classified through the first model according to the first image, and the accuracy of sleep classification can be improved.

Description

Technical Field

[0001] The present invention relates to the field of signal processing technology, and in particular to a state detection method, device, electronic device and storage medium based on electroencephalogram (EEG) signals. Background Art

[0002] Sleep staging is based on changes in physiological signals such as electroencephalogram (EEG), electrooculogram (EOG) and electromyogram (EMG), which divides the sleep process into different stages. This is of great significance for understanding sleep structure and its impact on health.

[0003] Traditional sleep staging relies on expert manual interpretation of polysomnography (PSG). This method is not only time-consuming and labor-intensive, but also susceptible to subjective factors, making it difficult to meet the needs of large-scale clinical screening. In recent years, machine learning methods have been introduced to the task of sleep state classification. Although they have achieved high classification accuracy on specific datasets, these methods often rely on manually designed features, have limited generalization capabilities, and are difficult to adapt to complex and changing real-world scenarios. This leads to inaccurate classification of EEG signals and an inability to accurately represent the sleep state corresponding to the EEG signals. Summary of the Invention

[0004] The present invention provides a state detection method, device, electronic device and storage medium based on EEG signals to solve the problem of inaccurate classification of EEG signals and inaccurate expression of the sleep state corresponding to the EEG signals due to insufficient generalization ability.

[0005] According to one aspect of the present invention, a method for detecting a state based on an electroencephalogram (EEG) signal is provided, comprising:

[0006] In response to the signal acquisition request, determining a first signal acquired by the EEG electrode; the first signal being an electrical signal acquired by the EEG electrode from the target subject while the target subject is sleeping;

[0007] Extracting and stitching spectrum features of the first signal to obtain a first image;

[0008] A first signal is classified according to the first image using a first model to obtain a first classification result; the first model is configured with a feature extraction layer and a time series signal capture layer; a processing layer is configured after each convolution layer in the feature extraction layer; the processing layer is used to normalize the output of each convolution layer; the first classification result is used to characterize the sleep state of the target object corresponding to the first signal.

[0009] According to another aspect of the present invention, there is provided a state detection device based on an electroencephalogram signal, comprising:

[0010] a first signal determination module, configured to respond to a signal acquisition request and determine a first signal acquired by the EEG electrodes; the first signal being an electrical signal acquired by the EEG electrodes from a target subject during sleep;

[0011] A first image determination module, configured to extract and stitch spectrum features of the first signal to obtain a first image;

[0012] A first classification result determination module is used to classify the first signal through a first model according to the first image to obtain a first classification result; the first model is configured with a feature extraction layer and a time series signal capture layer; each convolution layer in the feature extraction layer is configured with a processing layer; the processing layer is used to normalize the output of each convolution layer; the first classification result is used to characterize the sleep state of the target object corresponding to the first signal.

[0013] According to another aspect of the present invention, an electronic device is provided, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the state detection method based on EEG signals described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the state detection method based on EEG signals described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention responds to a signal acquisition request and determines the first signal collected by the EEG electrode; extracts and splices the spectral features of the first signal to obtain a first image. The determination of the first image can better characterize the dynamic fluctuations and non-stationary characteristics of the first signal; classifies the first signal through a first model based on the first image to obtain a first classification result. Since the first model introduces a processing layer, it can alleviate the internal covariance offset problem that may occur in the first model, thereby ensuring the accuracy of the first model's prediction. By extracting and splicing the spectral features of the first signal, this method can enable the obtained first image to better characterize the frequency characteristic changes of the first signal, and classify the sleep type to which the first signal belongs through the first model based on the first image, thereby improving the accuracy of sleep classification.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flow chart of a state detection method based on EEG signals provided by an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a first image provided by an embodiment of the present invention;

[0023] Figure 3 An architectural diagram of a first model provided by an embodiment of the present invention;

[0024] Figure 4 A classification confusion matrix of a first model prediction result provided in an embodiment of the present invention;

[0025] Figure 5 A CWT feature fusion graph provided by an embodiment of the present invention;

[0026] Figure 6 An MFCC feature fusion map provided by an embodiment of the present invention;

[0027] Figure 7 A schematic structural diagram of a state detection device based on EEG signals provided by an embodiment of the present invention;

[0028] Figure 8 A schematic structural diagram of an electronic device for implementing the state detection method based on EEG signals according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Figure 1 This is a flow chart of a method for detecting a state based on EEG signals provided by an embodiment of the present invention. This embodiment is applicable to the case of staging sleep based on EEG signals. The method can be performed by a state detection device based on EEG signals. The state detection device based on EEG signals can be implemented in the form of hardware and / or software. The state detection device based on EEG signals can be configured in any electronic device with network communication function. Figure 1 As shown, the method includes:

[0032] S110 , in response to a signal acquisition request, determining a first signal acquired by the EEG electrode; the first signal is an electrical signal acquired by the EEG electrode when the target object is sleeping.

[0033] The signal collection request is a signal collection instruction issued according to the signal collection requirement. The signal collection requirement is a collection requirement generated when a sleep state assessment is required.

[0034] Among them, the EEG electrodes contact the target object and transmit the weak potential difference (usually microvolt level) generated when the target object is in different activities to the amplifier, forming an electrical signal that can be recorded and analyzed.

[0035] The first signal is an electrical signal directly collected by the EEG electrodes or a signal obtained by processing the collected electrical signal. Furthermore, whether the collected signal needs to be processed depends on the frequency of the signal. For example, if the frequency of the collected signal is outside the range of 0.5-30 Hz, processing is required.

[0036] Specifically, the EEG electrodes respond to signal acquisition requests and collect signals from the target subject while they are asleep, generating a second signal. A determination is made as to whether the second signal needs to be processed based on the frequencies of the various signal features within the second signal. If so, the second signal is filtered and normalized to generate the first signal. If not, the second signal is normalized to generate the first signal.

[0037] Furthermore, the second signal is processed because the collected electrical signal may contain high-frequency electromyographic interference and low-frequency baseline drift, which will affect the judgment of the sleep state; the normalization processing is performed because the frequency of the collected signal is between 0.5-30Hz, and the frequency span is large, so normalization processing is required.

[0038] Furthermore, before the EEG electrode collects the first signal, the collection requirement is input into the input interface connected to the EEG electrode, a signal collection request is generated according to the collection requirement, and sent to the EEG electrode, which responds to the request and collects the signal.

[0039] S120 : Extract spectrum features and perform spectrum feature splicing on the first signal to obtain a first image.

[0040] Specifically, different spectrum feature acquisition methods are used to extract spectrum features from the first signal, and the obtained different spectrum features are spliced ​​along a time axis to obtain a first image.

[0041] Furthermore, different spectral feature acquisition methods include the Mel-Frequency Cepstral Coefficients (MFCC) method and the Continuous Wavelet Transform (CWT). The Mel-Frequency Cepstral Coefficients (MFCC) method is used to convert the first signal from a time-domain signal into a Mel-frequency signal. The Mel-frequency signal is a type of signal that can be perceived by the auditory system. The Continuous Wavelet Transform is used to obtain the time-frequency features of the first signal.

[0042] Furthermore, the steps for obtaining spectral features using Mel-Frequency Cepstral Coefficients (MFCCs) are as follows: first, pre-emphasize the first signal, perform frame segmentation and windowing after pre-emphasis, and perform Fourier transform to convert the first signal into a frequency domain signal. The spectrum of the frequency domain signal is modulo-squared to obtain a power spectrum. The power spectrum is filtered using a Mel filter bank and de-correlated using a discrete cosine transform to obtain a set of second images.

[0043] Furthermore, the steps of obtaining spectral features through continuous wavelet transform include: selecting a mother wavelet function based on the signal characteristics of the first signal, determining a scale range and a scale length based on the signal length of the first signal, selecting the first signal based on the scale length and the scale range, and convolving the signal within each scale with the mother wavelet function to obtain a time-frequency matrix. A set of second images is generated based on the time-frequency matrix.

[0044] Furthermore, the different spectral features obtained are spliced ​​according to the time axis to obtain the first image in the following steps: performing dimensional feature extraction on the first image obtained by MFCC extraction and the first image obtained by CWT extraction, ensuring that the number of feature frames is consistent, and splicing the extracted features in chronological order to obtain the first image.

[0045] For example, Figure 2 As shown in FIG, the first image after splicing is shown. As can be seen from the figure, it contains both time-frequency features and MFCC features.

[0046] Furthermore, the splicing of different spectral features can also be achieved through the attention mechanism. That is, dimensional features are extracted from the first image extracted by MFCC and the first image extracted by CWT, and the extracted features are fused through the attention mechanism.

[0047] Furthermore, the splicing of different spectral features may also be: determining a differential feature image according to the spectral features in the first image extracted by MFCC, and splicing the differential feature image with the first image extracted by MFCC and the first image extracted by CWT.

[0048] S130. Classify the first signal through the first model according to the first image to obtain a first classification result; the first model is configured with a feature extraction layer and a time series signal capture layer; a processing layer is configured after each convolution layer in the feature extraction layer; the processing layer is used to normalize the output of each convolution layer; the first classification result is used to characterize the sleep state of the target object corresponding to the first signal.

[0049] The feature extraction layer and the time series signal capture layer of the first model are connected through a Reshape layer. The Reshape layer adjusts the dimension of the output data without changing the output data content of the feature extraction layer.

[0050] Among them, the feature extraction layer consists of four stacked CNN convolution blocks. Each CNN convolution block is configured with a Dropout layer to inhibit the neurons of the next convolution block.

[0051] Each convolutional block consists of a two-dimensional convolutional layer, a LeakyReLU activation function layer, a max pooling layer, and a processing layer. Each layer uses a 2×2 convolution kernel. The processing layer normalizes the output of the convolutional layer, converting each convolutional layer output into a standard normal distribution with a mean of 0 and a variance of 1. This configuration of the processing layer is intended to mitigate the internal covariance shift that may occur during the training of the first model.

[0052] The dropout rate of the Dropout layer is set to 0.1 to improve the generalization ability of the model and prevent overfitting.

[0053] The time series signal capture layer can use an LSTM layer. The LSTM layer contains 256 hidden units, adopts a unidirectional structure, and outputs a total of 256-dimensional feature vectors for capturing the time series transition relationship between signals of different frequency bands in the first signal.

[0054] Furthermore, after the time series signal capture layer, a two-layer fully connected network performs classification mapping on the feature vectors output by the time series signal capture layer. The first fully connected layer maps the input to a 128-dimensional feature space and introduces nonlinear transformations. The second layer further compresses the feature vectors output by the time series signal capture layer into a 5-dimensional output space, corresponding to the five sleep states. The LogSoftmax function then outputs the probability distribution of each state, resulting in the sleep state of the first signal.

[0055] For example, the structure of the first model is as follows: Figure 3 As shown in the figure, the feature extraction layer and the time series signal capture layer are connected via the Reshap layer. The Reshap layer converts the output of the feature extraction layer into a 32×512 sequence format and expands it to a sequence dimension of length 1, which serves as the input to the time series signal layer. A two-layer fully connected network performs classification mapping on the feature vectors output by the time series signal layer. The first fully connected layer maps the input to a 128-dimensional feature space and introduces a nonlinear transformation to increase the dimensionality of the output features. The second layer further compresses the features into a 5-dimensional output space, corresponding to the five sleep state categories. The LogSoftmax function is then used to output the probability distribution of each sleep state, and the results are output based on the probability distribution.

[0056] The target subject's sleep state is divided into the following five categories: wakefulness (W), rapid eye movement (REM), and non-rapid eye movement (NREM). NREM sleep is further divided into three stages: N1 (light sleep), N2 (moderate sleep), and N3 (deep sleep). The above five sleep states are recorded as Class 0, Class 1, Class 2, Class 3, and Class 4 respectively.

[0057] Specifically, the first image is input into the first model, the feature extraction layer and the time series feature layer in the first model extract features from the first image, and the data of the time series feature layer is mapped through the fully connected layer to obtain a first classification result.

[0058] Furthermore, after obtaining the first classification result, the corresponding state stage is matched from the state correspondence according to the value of the first classification result, and the sleep state of the target object is determined according to the obtained state stage.

[0059] Furthermore, the state correspondence is determined by quantizing the sleep state of the target object to obtain quantized value ranges corresponding to the five sleep states, and then matching the obtained value ranges with the corresponding sleep states to obtain the state correspondence.

[0060] For example, the result diagram of the first model is as follows: Figure 4As shown in the figure, for Class0, 429 first signals were predicted correctly, 6 first signals were predicted as Class1, 76 first signals were predicted as Class2, 23 first signals were predicted as Class3, and 1 first signal was predicted as Class4. For Class1, 31 first signals were predicted as Class0, 219 first signals were predicted correctly, 14 first signals were predicted as Class2, 16 first signals were predicted as Class3, and 0 first signal was predicted as Class4. For Class2, 65 first signals were predicted as Class0, 28 first signals were predicted as Class1, 101 first signals were predicted correctly, 45 first signals were predicted as Class3, and 0 first signal was predicted as Class4. For Class 3, 18 first signals were predicted as Class 0, 29 first signals were predicted as Class 1, 28 first signals were Class 2, 711 first signals were predicted correctly, and 26 first signals were predicted as Class 4. For Class 4, 0 first signals were predicted as Class 0, 0 first signals were predicted as Class 1, 0 first signals were Class 2, 57 first signals were predicted as Class 3, and 156 first signals were predicted correctly. It can also be seen from the figure that the darker the color of the diagonal line, the higher the prediction accuracy. From this, it can be seen that the first model has a high accuracy rate for the classification of the first signal and can meet the needs of sleep staging.

[0061] Furthermore, if Figure 4 As shown, the first model performs best in stage N2 (Class 3), with the vast majority of samples correctly classified. However, in stage N1 (Class 2), there are many misclassifications, primarily misidentified as stage W or stage N2. Furthermore, some stage W samples are also misclassified as stage N1 or N2. However, the overall confusion matrix shows a strong main diagonal distribution trend, which meets the requirements for sleep staging.

[0062] Furthermore, the present application also establishes a control group based on the characteristic images. The characteristic images of the control group are: the fourth image and the fifth image. The fourth image is obtained by spatially stacking the first image obtained by CTW, such as Figure 5 The fifth image is obtained by fusion of spectrum features of the first image obtained by MFCC, as shown in Figure 6 The evaluation results of the classification accuracy of the first model for the fourth image and the fifth image are shown in Table 1.

[0063] Table 1 Classification evaluation results of different feature images

[0064]

[0065] As shown in Table 1, when using only MFCC features, the overall classification performance of the first model is relatively low, with an accuracy of 63.7% and an F1 score of 0.471. Specifically, the accuracy for the REM stage is only 0.275, and for the N1 stage is only 0.302, indicating that this feature is significantly insufficient in capturing and modeling the physiological characteristics of these stages. In contrast, using the CWT feature extraction method, the overall accuracy of the first model increased to 74.7%, with an F1 score of 0.691, demonstrating stronger classification capabilities. It showed particularly good discrimination results for the W stage (accuracy of 0.768) and the N1 stage (accuracy of 0.809).

[0066] Furthermore, by fusing MFCC and CWT features, the performance of the first model was further improved, reaching an overall accuracy of 77.1% and an F1 score of 0.718. Notably, in the REM stage, the F1 score increased from 0.755 for CWT alone to 0.832, an improvement of 0.077. Meanwhile, the classification accuracy in the N2 stage approached 0.954, making it the best performing class among all stages.

[0067] Optionally, in response to a signal acquisition request, determining a first signal acquired by an EEG electrode includes steps A1-A3:

[0068] Step A1: Acquire a second signal collected by the EEG electrodes; the second signal is an original electrical signal collected by the EEG electrodes from the target object during sleep.

[0069] Specifically, the EEG electrodes respond to signal collection requirements and collect signals from the target object while the target object is sleeping to obtain a second signal.

[0070] Step A2: filter the second signal to obtain a third signal.

[0071] Specifically, whether the second signal needs to be processed is determined based on the frequency of each signal feature in the second signal. If necessary, the second signal is filtered through a fourth-order Butterworth bandpass filter to remove high-frequency electromyographic interference (>30Hz) and low-frequency baseline drift (<0.5Hz), retaining the delta wave, theta wave, alpha wave and beta wave, and the remaining signal is used as the third signal.

[0072] Further, if not necessary, normalization processing is performed on the second signal to obtain the first signal.

[0073] Among them, the frequency range of delta wave is 0.5-4Hz; the frequency range of theta wave is 4-8Hz; the frequency range of alpha wave is 8-13Hz; the frequency range of beta wave is 13-30Hz.

[0074] Step A3: normalize the third signal to obtain the first signal.

[0075] Specifically, the third signal is normalized, and its amplitude is uniformly mapped to the interval [0, 1] to obtain the first signal.

[0076] Optionally, performing spectrum feature extraction and spectrum feature splicing on the first signal to obtain a first image includes steps B1-B2:

[0077] Step B1: extract spectrum features from the first signal to obtain at least one set of second images; the spectrum feature extraction is used to extract spectrum features from the first signal at time-frequency and Mel-frequency levels.

[0078] The Mel frequency is a nonlinear frequency scale based on the perceptual characteristics of the auditory system.

[0079] Specifically, different spectrum feature acquisition methods are used to extract spectrum features from the first signal, and the obtained different spectrum features are spliced ​​along a time axis to obtain a first image.

[0080] Furthermore, different spectrum feature acquisition methods include Mel-frequency cepstral coefficient method and continuous wavelet transform.

[0081] Furthermore, the steps for obtaining spectral features using Mel-Frequency Cepstral Coefficients (MFCCs) are as follows: first, pre-emphasize the first signal, perform frame segmentation and windowing after pre-emphasis, and perform Fourier transform to convert the first signal into a frequency domain signal. The spectrum of the frequency domain signal is modulo-squared to obtain a power spectrum. The power spectrum is filtered using a Mel filter bank and de-correlated using a discrete cosine transform to obtain a set of second images.

[0082] Furthermore, the step of obtaining spectral features through continuous wavelet transform includes: selecting a mother wavelet function based on the signal characteristics of the first signal, determining a scale range and a scale length based on the signal length of the first signal, selecting the first signal based on the scale length and the scale range, and convolving the signal within each scale with the mother wavelet function to obtain a time-frequency matrix. A set of second images is generated based on the time-frequency matrix.

[0083] Step B2: splicing at least one set of second images along the time axis to obtain a first image.

[0084] Specifically, dimensional features are extracted from at least one group of second images, and the number of feature frames is ensured to be consistent. The extracted features are spliced ​​in chronological order to obtain the first image.

[0085] Optionally, performing spectrum feature extraction on the first signal to obtain at least one set of second images includes steps C1-C4:

[0086] Step C1: pre-emphasize the first signal to obtain a fourth signal; the pre-emphasis is used to increase the high-frequency signal segment of the first signal.

[0087] Specifically, pre-emphasis processing is performed on the first signal according to the pre-emphasis coefficient to obtain the fourth signal.

[0088] Furthermore, the fourth signal can be expressed as:

[0089] y(n)=x(n)-αx(n-1);

[0090] Wherein, y(n) is the fourth signal; x(n) is the first signal; α is the pre-emphasis coefficient, which is 0.95 or 0.97.

[0091] Step C2: perform frame division and windowing processing on the fourth signal, and perform Fourier transform on the fourth signal to obtain a fifth signal; the fifth signal is a frequency domain signal.

[0092] Specifically, a frame length and a frame shift are determined based on the fourth signal, the fourth signal is framed according to the frame length and the frame shift, and the framed signal is windowed using a window function. After the processing is completed, Fourier transform is performed to obtain the fifth signal.

[0093] As mentioned above, the framing process is to divide the first signal into short signals and ensure that the signal in each signal segment is smooth; the windowing process is to prevent the problem of spectrum leakage in the signal.

[0094] The frame length may be 20-40 ms, and the frame shift may be 10-15 ms.

[0095] Step C3: determine a power spectrum according to the fifth signal, and filter the power spectrum to obtain a sixth signal.

[0096] Specifically, the fifth signal is calculated according to a power spectrum calculation formula to obtain a power spectrum, and the obtained power spectrum is filtered through a Mel filter to obtain a sixth signal.

[0097] Furthermore, the power spectrum calculation formula can be expressed as:

[0098]

[0099] Wherein, N is the number of Fourier transform points; y(k) is the Fourier transform of the fourth signal; P(k) is the power spectrum.

[0100] Wherein, the Mel filter converts the linear power spectrum into the feature of the Mel scale by simulating the perception characteristics of the auditory system to different frequencies.

[0101] Further, the Mel filter can be expressed as:

[0102]

[0103] Step C4, performing discrete cosine transform on the sixth signal and generating a set of second images.

[0104] Wherein, the discrete cosine transform is used to remove the correlation in the sixth signal caused by the parameters in the Mel filter.

[0105] Specifically, the discrete cosine transform is used to process the sixth signal to remove the influence of the correlation of the coefficients in the Mel filter in the filtering process, and a set of first images is generated according to the processed signal.

[0106] Optionally, performing spectral feature extraction on the first signal to obtain at least one set of second images, including steps D1-D3:

[0107] Step D1, performing translation and scaling processing on the mother wavelet function to obtain a wavelet function family.

[0108] Specifically, the mother wavelet function is selected according to the signal characteristics of the first signal, and the translation and scaling processing is performed on the mother wavelet function to obtain a wavelet function family.

[0109] Wherein, the wavelet function family can be expressed as:

[0110]

[0111] Wherein, w0 is the center frequency of the wavelet.

[0112] Step D2, determining a time-frequency matrix according to the first signal and the wavelet function family.

[0113] Specifically, the first signal obtained is convolved with the wavelet function family to obtain a time-frequency matrix.

[0114] Wherein, the time-frequency matrix can be expressed as:

[0115]

[0116] Wherein, f(t) represents the first signal, ψ represents the mother wavelet function in the wavelet function family, a is a scale parameter that determines the scaling scale of the wavelet function, and b is a translation parameter that changes the value to make the function translate on the coordinate axis.

[0117] Step D3: Generate a set of second images according to the time-frequency matrix.

[0118] Specifically, a set of second images is generated using a visualization tool according to the acquired time-frequency matrix.

[0119] Furthermore, there is no limitation on the visualization tool, and the visualization tool can be selected based on the characteristics to be displayed. For example, if a heat map is required to display spectrum characteristics, the matplotlib tool can be used.

[0120] Optionally, the first model is constructed using at least one of the following methods, including steps E1-E3:

[0121] Step E1, determine the third image and the second classification result; the third image is a mosaic of the spectral characteristics of the seventh signal; the second classification result is used to characterize the sleep state of the target object corresponding to the seventh signal; the seventh signal is an electrical signal obtained by collecting the target object's sleep by the EEG electrodes at a historical moment; the historical moment is the moment before the second signal is collected.

[0122] Among them, the sixth signal is the electrical signal obtained by the EEG electrodes collecting data from the target object at the historical moment.

[0123] Specifically, historical signals collected by the EEG electrodes at historical moments are acquired, and based on the frequencies of the various signal features within the acquired electrical signals, a determination is made as to whether the acquired signals need to be processed. If so, the acquired signals are filtered and normalized to obtain a seventh signal. If not, the acquired signals are directly normalized to obtain the seventh signal; a third image is generated based on the seventh signal in the same manner as the first image, which will not be further described herein. A second classification result corresponding to the seventh signal is obtained from a storage device.

[0124] The storage device is used to store the seventh signal, the second classification result, and the sleep state corresponding to the seventh signal. The storage device and the EEG electrodes are configured in the same acquisition device.

[0125] Step E2: Input the third image into the second model to perform signal classification on the seventh signal to obtain a third classification result; the second model has the same network structure as the first model, but different model parameters.

[0126] Specifically, the third image is input into the second model, the feature extraction layer and the time series feature layer in the second model extract features from the third image, and the data of the time series feature layer is mapped through the fully connected layer to obtain a third classification result.

[0127] Step E3: Determine the first data according to the third classification result and the second classification result.

[0128] Specifically, the number of correctly classified samples and the number of incorrectly classified samples in the third classification result are determined based on the third classification result and the second classification result. Error assessment is performed based on the number of correctly classified samples and the number of incorrectly classified samples to obtain the first data.

[0129] Furthermore, the first data is determined by accuracy (Accuracy) and F1 score (F1 Score).

[0130] The accuracy can be expressed by the following formula:

[0131]

[0132] Among them, the F1 score can be expressed by the following formula:

[0133]

[0134] Among them, TP represents the number of samples that are actually positive and correctly predicted to be positive, FN represents the number of samples that are actually positive but incorrectly predicted to be negative, FP represents the number of samples that are actually negative but incorrectly predicted to be positive, and TN represents the number of samples that are actually negative and correctly predicted to be negative.

[0135] Step E4: Modify the second model according to the first data to obtain the first model.

[0136] Specifically, the parameters of different layers in the second model are corrected according to the first data, and the third image is predicted based on the corrected model. It is judged whether the preset requirements are met based on the prediction results. If so, the model corresponding to the parameters that meet the preset requirements is used as the first model; if not, the iteration is continued until the preset requirements are met.

[0137] The technical solution of this embodiment responds to a signal acquisition request and determines the first signal collected by the EEG electrode; extracts and splices the spectral features of the first signal to obtain a first image, and the determination of the first image better characterizes the dynamic fluctuations and non-stationary characteristics of the first signal; classifies the first signal through a first model based on the first image to obtain a first classification result. Since the first model introduces a processing layer, it can alleviate the internal covariance offset problem that may occur in the first model, thereby ensuring the accuracy of the first model's prediction. By extracting and splicing the spectral features of the first signal, this method can enable the obtained first image to better characterize the frequency characteristic changes of the first signal, and classify the sleep type to which the first signal belongs through the first model based on the first image, thereby improving the accuracy of sleep classification.

[0138] Figure 7This is a schematic diagram of the structure of a state detection device based on EEG signals provided by an embodiment of the present invention. This embodiment is applicable to the case of classifying sleep stages based on EEG signals. The state detection device based on EEG signals can be implemented in the form of hardware and / or software. The state detection device based on EEG signals can be configured in any electronic device with network communication function. Figure 7 As shown, the apparatus includes: a first signal determination module 210, a first image determination module 220 and a first classification result determination module 230, wherein:

[0139] First signal determination module 210: for responding to the signal acquisition request and determining a first signal acquired by the EEG electrodes; the first signal is an electrical signal acquired by the EEG electrodes from the target subject during sleep;

[0140] The first image determination module 220 is configured to extract and combine spectrum features of the first signal to obtain a first image.

[0141] First classification result determination module 230: used to classify the first signal through the first model according to the first image to obtain a first classification result; the first model is configured with a feature extraction layer and a time series signal capture layer; each convolution layer in the feature extraction layer is configured with a processing layer; the processing layer is used to normalize the output of each convolution layer; the first classification result is used to characterize the sleep state of the target object corresponding to the first signal.

[0142] Optionally, the first signal determination module 210 includes:

[0143] A second signal determination unit is used to obtain a second signal collected by the EEG electrodes; the second signal is an original electrical signal collected by the EEG electrodes from the target object during sleep;

[0144] A third signal determining unit is configured to filter the second signal to obtain a third signal;

[0145] The first signal determining unit is configured to perform normalization processing on the third signal to obtain the first signal.

[0146] Optionally, the first image determination module 220 includes:

[0147] The second image determination unit is configured to perform spectrum feature extraction on the first signal to obtain at least one set of second images; the spectrum feature extraction is configured to perform spectrum feature extraction on the first signal from a time-frequency perspective and a Mel-frequency perspective;

[0148] The first image determining unit is configured to splice at least one set of second images along a time axis to obtain a first image.

[0149] Optionally, the second image determination unit includes:

[0150] a fourth signal determination subunit, configured to perform pre-emphasis on the first signal to obtain a fourth signal; the pre-emphasis is configured to increase a high frequency signal segment of the first signal;

[0151] a fifth signal determination subunit, configured to perform framing and windowing processing on the fourth signal, and perform Fourier transform on the fourth signal to obtain a fifth signal; the fifth signal is a frequency domain signal;

[0152] a sixth signal determining subunit, configured to determine a power spectrum according to the fifth signal, and filter the power spectrum to obtain a sixth signal;

[0153] The second image determination subunit is configured to perform discrete cosine transform on the sixth signal and generate a set of second images.

[0154] Optionally, the second image determination unit includes:

[0155] The wavelet function family determines the subunit, which is used to perform translation and scaling processing based on the mother wavelet function to obtain the wavelet function family;

[0156] a time-frequency matrix determining subunit, configured to determine a time-frequency matrix according to the first signal and a wavelet function family;

[0157] The second image determination subunit is configured to generate a set of second images according to the time-frequency matrix.

[0158] Optionally, the first classification result determination module 230 includes:

[0159] a data determination unit, configured to determine a third image and a second classification result; the third image being a mosaic of spectral features of the seventh signal; the second classification result being used to characterize the sleep state of the target subject corresponding to the seventh signal; the seventh signal being an electrical signal acquired by an EEG electrode of the target subject while sleeping at a historical moment; the historical moment being a moment before the acquisition of the second signal;

[0160] a third classification result determining unit, configured to input the third image into the second model to perform signal classification on the seventh signal to obtain a third classification result; the second model and the first model have the same network structure but different model parameters;

[0161] a first data determining unit, configured to determine first data according to the third classification result and the second classification result;

[0162] The first model determining unit is configured to modify the second model according to the first data to obtain the first model.

[0163] The state detection device based on EEG signals provided in the embodiments of the present invention can execute the state detection method based on EEG signals provided in any of the above-mentioned embodiments of the present invention, and has the corresponding functions and beneficial effects of executing the state detection method based on EEG signals. For detailed processes, please refer to the relevant operations of the state detection method based on EEG signals in the above-mentioned embodiments.

[0164] Figure 8 Schematic diagram of the structure of an electronic device for implementing the state detection method based on EEG signals of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0165] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0166] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0167] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the state detection method based on EEG signals.

[0168] In some embodiments, the state detection method based on EEG signals can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the state detection method based on EEG signals described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the state detection method based on EEG signals in any other appropriate manner (for example, by means of firmware).

[0169] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0170] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0171] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0173] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0174] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0175] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0176] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A state detection method based on EEG signals, characterized in that: include: In response to the signal acquisition request, determining a first signal acquired by the EEG electrode; The first signal is an electrical signal obtained by collecting the target object during sleep through the EEG electrodes; Extracting and stitching spectrum features of the first signal to obtain a first image; A first signal is classified according to the first image using a first model to obtain a first classification result; the first model is configured with a feature extraction layer and a time series signal capture layer; a processing layer is configured after each convolution layer in the feature extraction layer; the processing layer is used to normalize the output of each convolution layer; the first classification result is used to characterize the sleep state of the target object corresponding to the first signal.

2. The method according to claim 1, characterized in that The step of responding to the signal acquisition request and determining the first signal acquired by the EEG electrode includes: Acquiring a second signal collected by the EEG electrode; the second signal is an original electrical signal collected by the EEG electrode from the target object during sleep; performing filtering on the second signal to obtain a third signal; Normalization is performed on the third signal to obtain a first signal.

3. The method according to claim 1, characterized in that The extracting and splicing spectrum features of the first signal to obtain a first image includes: Performing spectrum feature extraction on the first signal to obtain at least one set of second images; the spectrum feature extraction is used to extract spectrum features of the first signal from time-frequency and Mel-frequency; The at least one set of second images are spliced ​​together according to a time axis to obtain a first image.

4. The method according to claim 3, characterized in that The extracting spectrum features of the first signal to obtain at least one set of second images includes: Pre-emphasize the first signal to obtain a fourth signal; the pre-emphasis is used to increase the high-frequency signal segment of the first signal; performing frame division and windowing processing on the fourth signal, and performing Fourier transform on the fourth signal to obtain a fifth signal; wherein the fifth signal is a frequency domain signal; determining a power spectrum according to the fifth signal, and filtering the power spectrum to obtain a sixth signal; The sixth signal is discrete cosine transformed to generate a set of second images.

5. The method according to claim 3, characterized in that The extracting spectrum features of the first signal to obtain at least one set of second images includes: According to the mother wavelet function, translation and expansion processing are performed to obtain a wavelet function family; determining a time-frequency matrix according to the first signal and the wavelet function family; A set of second images is generated according to the time-frequency matrix.

6. The method according to claim 1, characterized in that The first model is constructed using at least one of the following methods: Determine a third image and a second classification result; the third image is a mosaic of spectral features of the seventh signal; the second classification result is used to characterize the sleep state of the target subject corresponding to the seventh signal; the seventh signal is an electrical signal obtained by EEG electrodes while the target subject is sleeping at a historical moment; the historical moment is a moment before the second signal is collected; Inputting the third image into a second model to perform signal classification on the seventh signal to obtain a third classification result; the second model has the same network structure as the first model but different model parameters; determining first data according to the third classification result and the second classification result; The second model is modified according to the first data to obtain a first model.

7. A state detection device based on EEG signals, characterized in that: include: a first signal determination module, configured to respond to a signal acquisition request and determine a first signal acquired by the EEG electrode; The first signal is an electrical signal obtained by collecting the target object during sleep through the EEG electrodes; A first image determination module is configured to extract and stitch spectrum features of the first signal to obtain a first image; A first classification result determination module is used to classify the first signal through a first model according to the first image to obtain a first classification result; the first model is configured with a feature extraction layer and a time series signal capture layer; each convolution layer in the feature extraction layer is configured with a processing layer; the processing layer is used to normalize the output of each convolution layer; the first classification result is used to characterize the sleep state of the target object corresponding to the first signal.

8. The device according to claim 7, characterized in that The first image determination module includes: A second image determination unit is configured to perform spectrum feature extraction on the first signal to obtain at least one set of second images; the spectrum feature extraction is configured to perform spectrum feature extraction on the first signal from a time-frequency perspective and a Mel-frequency perspective; The first image determining unit is configured to splice the at least one set of second images along a time axis to obtain a first image.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the state detection method based on electroencephalogram signals according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the state detection method based on EEG signals according to any one of claims 1 to 6 when executed.

Images