Electroencephalogram signal prediction method and equipment based on neural network

Abstract

The invention provides an electroencephalogram signal prediction method and device based on a neural network, and relates to the field of signal processing, and the method comprises the following steps: obtaining and converting a to-be-detected electroencephalogram signal into a power spectrum image, and after obtaining a target Mel-frequency cepstrum coefficient of the power spectrum image, obtaining a target Mel-frequency cepstrum coefficient of the power spectrum image; and inputting the target Mel-frequency cepstrum coefficient into the trained neural network model so as to obtain weighted features through an attention mechanism under the neural network model. And finally, classifying the weighted features by using multiple groups of sample data, and taking a classification result as a prediction result of the electroencephalogram signal to be detected. According to the method, the Mel-frequency cepstrum coefficient and the attention mechanism are fused, and the Mel-frequency cepstrum coefficient is utilized to emphasize low-frequency spectrum features, so that the neural network based on the attention mechanism can better identify low-frequency spectrum information, and the prediction accuracy is improved; and meanwhile, the dimensionality of the features is greatly reduced by using logarithm operation and discrete cosine transform of the Mel-frequency cepstrum coefficient, and the recognition speed of the deep neural network is further improved.

Description

Technical Field

[0001] The present invention relates to the field of signal processing, and in particular, to an electroencephalogram signal prediction method and device based on a neural network. Background Art

[0002] Epilepsy is a common brain nervous system disease, which results from abnormal discharges of brain neurons and causes transient brain dysfunction. Currently, the prediction of epilepsy in clinical practice relies on the electroencephalogram monitoring of patients. Usually, it mainly relies on the time-frequency domain features of electroencephalogram signals, such as EWT, STFT, and wavelet transform, etc. However, these techniques do not consider the characteristics that the electroencephalogram signal itself has a low bandwidth, while the signature signals of epileptic seizures are usually in the low frequency range (0 - 30 Hz).

[0003] With the iterative development of computing technology, epilepsy seizure prediction methods based on machine learning and deep learning have gradually emerged. These methods analyze electroencephalogram data, predict and classify the stages of epilepsy in advance, so as to provide timely early warnings and intervention measures for patients. However, the classifier structures of existing machine learning techniques are simple, with poor robustness, and cannot effectively and completely adapt to the information of large amounts of data, resulting in low prediction classification accuracy of electroencephalogram signals.

[0004] Based on this, there is an urgent need for a prediction classification method for electroencephalogram signals, which can greatly reduce the dimension of data features and improve the recognition speed and accuracy of deep neural networks. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an electroencephalogram signal prediction classification method and device, which can greatly reduce the dimension of data features and improve the recognition speed and accuracy of deep neural networks by fusing Mel-frequency cepstral coefficients and attention mechanisms.

[0006] To achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows:

[0007] In a first aspect, the present invention provides an electroencephalogram signal prediction method based on a neural network, including the following steps:

[0008] Obtain an electroencephalogram signal to be measured, and convert the electroencephalogram signal to be measured into a power spectrum image;

[0009] Obtain the target Mel-frequency cepstral coefficients of the power spectrum image;

[0010] Input the target Mel-frequency cepstral coefficients into a trained neural network model to obtain weighted features; wherein, the trained neural network model includes a convolutional attention module;

[0011] Classify the weighted features using multiple groups of sample data, and use the classification result as the prediction result of the electroencephalogram signal to be measured.

[0012] Optionally, the steps of obtaining the target Mel-frequency cepstral coefficients of the power spectrum image include:

[0013] Converting the power spectrum image into a Mel-spectrum using a Mel filter bank;

[0014] Performing a discrete cosine transform on the Mel-spectrum to obtain multiple Mel-frequency cepstral coefficients;

[0015] Selecting the target Mel-frequency cepstral coefficients with orders less than a set value from the multiple Mel-frequency cepstral coefficients.

[0016] Optionally, the target Mel-frequency cepstral coefficients are expressed as:

[0017]

[0018] MFCC i =[c i (1),c i (2),…,c i (L)];

[0019] Wherein, MFCC i is the target Mel-frequency cepstral coefficient; c i (n) is the Mel-frequency cepstral coefficient corresponding to the nth time frame; M is the number of filters in the Mel filter bank; log(.) is the logarithmic function; L is the number of target Mel-frequency cepstral coefficients; cos(.) is the cosine function.

[0020] Optionally, the convolutional attention module includes a channel attention module and a spatial attention module; the steps of inputting the target Mel-frequency cepstral coefficients into a trained neural network model to obtain weighted features include:

[0021] Performing channel weighting on the target Mel-frequency cepstral coefficients using the channel attention module to obtain initial weighted features;

[0022] Performing spatial weighting on the initial weighted features using the spatial attention module to obtain the final weighted features.

[0023] Optionally, the weighted features are expressed as:

[0024]

[0025] Wherein, F out is the weighted feature; F c is the initial weighted feature; M c is the channel attention feature; M s is the spatial attention feature; is matrix multiplication.

[0026] Optionally, the trained neural network model further includes a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, and a fully connected layer;

[0027] Among them, the first convolutional layer, the first downsampling layer, the channel attention module, the spatial attention module, the second convolutional layer, the second downsampling layer, and the fully connected layer are connected in sequence, and the output of the fully connected layer is used as the output of the trained neural network model.

[0028] Optionally, before the step of converting the electroencephalogram signal to be measured into a power spectrum image, it further includes:

[0029] Preprocess the electroencephalogram signal to be measured to filter out the high-frequency components in the electroencephalogram signal to be measured.

[0030] Optionally, the design parameters of the Mel filter bank satisfy:

[0031]

[0032] Among them, mel(.) is the Mel frequency; f is the linear frequency; log(.) is the logarithmic function.

[0033] Optionally, the Mel spectrum satisfies:

[0034]

[0035] Among them, S i (m) is the Mel frequency corresponding to the m-th filter in the Mel filter bank; M is the number of filters in the Mel filter bank; P i (k) is the power spectrum; H m (k) is the frequency response corresponding to the m-th filter; N is the total number of frequency points k under the power spectrum P i (k).

[0036] In a second aspect, the present invention further provides an electronic device, including a processor and a memory, the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the neural network-based electroencephalogram signal prediction method according to any one of the first aspects above.

[0037] The neural network-based electroencephalogram signal prediction method and device provided by the present invention have the following beneficial effects:

[0038] The present invention acquires an electroencephalogram (EEG) signal to be measured, converts the EEG signal to be measured into a power spectrum image, and after obtaining the target Mel-frequency cepstral coefficients of the power spectrum image, inputs the target Mel-frequency cepstral coefficients into a trained neural network model. Through the attention mechanism in the neural network model, that is, a convolutional attention module, weighted features are obtained. Finally, multiple groups of sample data are used to classify the weighted features, and the classification result is used as the prediction result of the EEG signal to be measured. By fusing the Mel-frequency cepstral coefficients with the attention mechanism, the present invention uses the Mel-frequency cepstral coefficients to emphasize the spectral features of low frequencies, enabling the neural network based on the attention mechanism to better identify the information of low-frequency spectra, thereby improving the prediction accuracy. At the same time, the logarithmic operation and discrete cosine transform of the Mel-frequency cepstral coefficients greatly reduce the dimension of the features, further improving the recognition speed of the deep neural network.

[0039] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, detailed descriptions are as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the accompanying drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0041] Figure 1 Shows the flowchart of the steps of the EEG signal prediction method based on a neural network provided by an embodiment of the present invention;

[0042] Figure 2 Shows the sub-step flowchart of step 100 in an embodiment of the present invention;

[0043] Figure 3 Shows the sub-step flowchart of step 102 in an embodiment of the present invention;

[0044] Figure 4 Shows the sub-step flowchart of step 103 in an embodiment of the present invention;

[0045] Figure 5 Shows the spectrogram of the power spectrum in an embodiment of the present invention;

[0046] Figure 6 Shows the sub-step flowchart of step 200 in an embodiment of the present invention;

[0047] Figure 7 Shows the frequency design diagram of the Mel filter bank in an embodiment of the present invention;

[0048] Figure 8Shows the spectrogram of the target Mel-frequency cepstral coefficients in the embodiments of the present invention;

[0049] Figure 9 Shows the structural schematic diagram of the trained neural network model in the embodiments of the present invention;

[0050] Figure 10 Shows the sub-step flowchart of step 300 in the embodiments of the present invention.

[0051] Icons: 100 - trained neural network model; 101 - convolutional attention module; 102 - channel attention module; 103 - spatial attention module; 104 - first convolutional layer; 105 - first downsampling layer; 106 - second convolutional layer; 107 - second downsampling layer; 108 - fully connected layer. Detailed implementation manners

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Usually, the components of the embodiments of the present invention described and shown herein can be arranged and designed in various different configurations.

[0053] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed present invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0054] It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0055] The classifier structure of machine learning in the prior art is simple, has poor robustness, and cannot effectively and completely adapt to the information of a large amount of data, resulting in a low prediction classification accuracy of electroencephalogram signals.

[0056] Based on this, the present invention provides a method for predicting and classifying electroencephalogram (EEG) signals, which greatly reduces the dimension of data features and improves the recognition speed and accuracy of deep neural networks.

[0057] Please refer to Figure 1 , Figure 1 which shows a flowchart of steps of the EEG signal prediction method based on a neural network in the present invention. The EEG signal prediction method includes the following steps:

[0058] Step 100: Obtain the EEG signal to be measured and convert the EEG signal to be measured into a power spectrum image.

[0059] Step 200: Obtain the target Mel-frequency cepstral coefficients of the power spectrum image.

[0060] Step 300: Input the target Mel-frequency cepstral coefficients into the trained neural network model to obtain weighted features.

[0061] Among them, the trained neural network model includes a convolutional attention module.

[0062] Step 400: Classify the weighted features using multiple groups of sample data and use the classification result as the prediction result of the EEG signal to be measured.

[0063] In this embodiment, a prediction scheme for EEG signals that combines Mel-frequency cepstral coefficients and a neural network model based on the attention mechanism is provided. The Mel-frequency cepstral coefficients can be used to emphasize the features of EEG signals through non-linear changes, especially the spectral features in the low-frequency band of EEG signals, enabling the neural network model to better identify low-frequency signals and improve the prediction accuracy. At the same time, the Mel-frequency cepstral coefficients can also greatly reduce the dimension of EEG signal features through logarithmic operations and discrete cosine transforms, greatly improving the recognition speed of deep neural networks.

[0064] Further, on the basis of Figure 1 , please refer to Figure 2 , Figure 2 which shows a flowchart of steps of Step 100 in the present invention; Step 100 in this embodiment further includes Step 101 to Step 103.

[0065] Step 101: Obtain the EEG signal to be measured.

[0066] Step 102: Preprocess the EEG signal to be measured to filter out high-frequency components in the EEG signal to be measured.

[0067] Step 103: Convert the preprocessed EEG signal to be measured into a power spectrum image.

[0068] It should be noted that the preprocessing method in this embodiment does not limit step 102, as long as it can reduce the power frequency signal interference, eye movement and muscle movement crosstalk, and unnecessary high-frequency components in the obtained electroencephalogram (EEG) signals to be measured. In this embodiment, the high-frequency components can be signal components greater than 30 Hz.

[0069] In a possible implementation manner, please, on the basis of Figure 2 , refer to Figure 3 , Figure 3 which shows a flowchart of a step of step 102 in an embodiment of the present invention; the preprocessing method of this step 102 includes steps 1021 to 1025.

[0070] Step 1021: Segment the EEG signals to be measured.

[0071] In this embodiment, the EEG signals to be measured are divided into segments with a preset time length. For example, segments of about 3 seconds are used to improve the subsequent acquisition speed of Mel frequency cepstral coefficients. Assume that the signal to be measured is represented as x(t), where 0 ≤ t ≤ T (T is the sampling period); for the interval corresponding to t 0 ~t 0 +Δt, the segmented EEG signals to be measured can be represented as

[0072] where satisfies:

[0073] In the formula, t 0 is the starting time of segmentation; Δt is the preset time length.

[0074] Step 1022: Separate the independent source signals from the segmented EEG signals to be measured.

[0075] This embodiment can reduce the noise in the EEG signal acquisition process through the above operations, restore the real EEG signals, and improve the signal accuracy.

[0076] Assume that the observed signals are N-channel signals. The observed signal x(t) (i.e., the segmented EEG signals to be measured above) can be represented as: x(t) = [x 1 (t), x 2 (t), …, x n (t)] T ; these signals are the linear mixture of N independent source signals s(t) = [s 1 (t), s 2 (t), …, s N (t)] T . According to the corresponding relationship between the above observed signals and independent source signals, a separation matrix W can be constructed such that the estimated source signal y(t) satisfies: y(t) = Wx(t).

[0077] Step 1023: Centralize the observed signal to make its mean zero.

[0078] Step 1024: whiten the centralized observation signal so that its covariance matrix becomes a unit matrix.

[0079] Among them, the observed signal z(t) after whitening can satisfy:

[0080] z(t)=Vx(t).

[0081] Where V is the whitening matrix.

[0082] Step 1025, target optimization is performed on the observed signal after whitening processing, and the separation matrix is ​​updated by an iterative ascent method; and after each iteration, the separation matrix is ​​orthogonalized until the separation matrix converges to obtain the preprocessed EEG signal to be measured.

[0083] The calculation process of updating the separation matrix by the iterative ascent method in this embodiment can be expressed as:

[0084]

[0085] Among them, η is the learning rate; J(W) is the negative entropy.

[0086] The calculation process of orthogonalizing the separation matrix in this embodiment can be expressed as:

[0087] W←(WW T ) -1 / 2W .

[0088] Based on the above steps, this embodiment can extract as many independent components as possible from the EEG signal to be tested, so that the EEG signal to be tested is closer to the electrical signal of the real brain area.

[0089] In one possible implementation, Figure 2 Based on the reference Figure 4 , Figure 4 A step flow chart of step 103 in an embodiment of the present invention is shown; the preprocessing method of step 103 in this embodiment includes steps 1031 to 1034.

[0090] Step 1031: pre-emphasize the pre-processed EEG signal to be tested.

[0091] Based on this, this embodiment can improve the high-frequency part of the signal by pre-emphasis, so that the spectrum corresponding to the pre-processed EEG signal to be measured is flatter and maintained in the entire frequency band from the low frequency band to the high frequency end.

[0092] In this embodiment, pre-emphasis can compensate for the high-frequency components in the input signal at the signal transmitting end. This preprocessing process can be expressed as:

[0093] x pre (t) = x(t) - αx(t - 1);

[0094] where, x pre (t) is the electroencephalogram (EEG) signal to be measured after pre-emphasis, α is the pre-emphasis coefficient, and x(t - 1) is the EEG signal at the previous time point.

[0095] Step 1032: Frame the pre-emphasized EEG signal to be measured.

[0096] In this embodiment, the above-mentioned EEG signal to be measured is divided into short-time frames. Each time frame may include N sampling points, and the frame shift is M sampling points. Based on this, the EEG signal to be measured x i (n) at the i-th frame can be expressed as: x i (n) = x pre (t i + n), n = 0, 1, …, N - 1.

[0097] Step 1033: Add a Hamming window to the framed EEG signal to be measured.

[0098] In this embodiment, a Hamming window is used to window the framed EEG signal to be measured. Among them, the number of points of the Hamming window is half of the sampling rate.

[0099] Step 1034: Perform a fast Fourier transform on the windowed EEG signal to be measured to obtain a frequency spectrum image, and obtain a power spectrum image based on the frequency spectrum image.

[0100] Among them, please refer to Figure 5 , Figure 5 which shows a schematic diagram of the power spectrum image of this embodiment. This power spectrum image P i (k) is expressed as: P i (k) = |X i (k)| 2 ;

[0101] In the formula, X i (k) is the frequency spectrum image, which satisfies:

[0102]

[0103] On the basis of Figure 1 , please refer to Figure 6 , Figure 6 which shows the flowchart of the sub-steps of step 200 in the embodiment of the present invention; step 200 in this embodiment includes steps 201 - step 203.

[0104] Step 201: Convert the power spectrum image into a Mel spectrum using a Mel filter bank.

[0105] Step 202: Perform a discrete cosine transform on the Mel spectrum to obtain multiple Mel frequency cepstral coefficients.

[0106] Step 203: Select target Mel frequency cepstral coefficients with orders less than a set value from the multiple Mel frequency cepstral coefficients.

[0107] In this embodiment, the Mel frequency cepstral coefficients are used to emphasize the spectral characteristics of the low-frequency band in the EEG signal. At the same time, based on the spectral characteristics of the low-frequency band in the EEG signal, the recognition speed and accuracy of the deep neural network are greatly improved.

[0108] The following will detail the process of obtaining the target Mel frequency cepstral coefficients in this embodiment.

[0109] In a possible implementation manner, step 201 of converting the power spectrum image into a Mel spectrum using a Mel filter bank is specifically as follows:

[0110] Map the above power spectrum image to the Mel frequency scale and weight the power spectrum image using a set of triangular filters.

[0111] Furthermore, to ensure the realization of emphasizing the spectral characteristics of the low-frequency band by the Mel frequency cepstral coefficients, a Mel filter bank applicable to low-frequency signals can be set up to effectively filter out high-frequency components. The design parameters of this Mel filter bank satisfy:

[0112]

[0113] where mel(.) is the Mel frequency; f is the linear frequency; log(.) is the logarithmic function.

[0114] In a possible implementation manner, please refer to Figure 7 , Figure 7 which shows the frequency design diagram of the Mel filter bank in this embodiment; the Mel filter bank in this embodiment may include M (M = 32) triangular filters. Among them, the number of triangular filters may be the same as the number of critical bands, such as the number of sampling points of the above-mentioned frame shifting. In this embodiment, the center frequencies of each triangular filter are evenly distributed on the Mel scale, and the interval between the center frequencies of each triangular filter decreases as the m value decreases and increases as the m value increases.

[0115] Correspondingly, the frequency response H m (k) of the m-th filter can be expressed as:

[0116]

[0117] Where f(k) is the linear frequency at the k-th frequency point; f(m + 1) is the center frequency of the (m + 1)-th filter; f(m - 1) is the center frequency of the (m - 1)-th filter; and f(m) is the center frequency of the m-th filter.

[0118] Based on this, the corresponding Mel spectrum in this embodiment can be expressed as:

[0119]

[0120] Where S i (m) is the Mel frequency corresponding to the m-th filter in the Mel filter bank; M is the number of filters in the Mel filter bank; P i (k) is the power spectrum; H m (k) is the frequency response corresponding to the m-th filter; and N is the total number of frequency points k under the power spectrum P i (k).

[0121] Step 202 then performs a discrete cosine transform calculation on the above Mel spectrum S i (m) to obtain Mel frequency cepstral coefficients.

[0122] Further, for the convenience of discrete cosine transform calculation, the Mel spectrum S i (m) can be first logarithmically operated. After obtaining the logarithmic Mel spectrum, a discrete cosine transform calculation is performed on the logarithmic Mel spectrum, and then multiple Mel frequency cepstral coefficients can be obtained.

[0123] Subsequently, the target Mel frequency cepstral coefficients with orders less than the set value can be selected from multiple Mel frequency cepstral coefficients. In this embodiment, the target Mel frequency cepstral coefficients with orders less than the set value can be the first L and the Mel frequency cepstral coefficients corresponding to the lower orders among multiple Mel frequency cepstral coefficients.

[0124] Further, the target Mel frequency cepstral coefficients in this embodiment are expressed as:

[0125]

[0126] MFCC i = [c i (1), c i (2), …, c i (L)];

[0127] Where MFCC i is the target Mel frequency cepstral coefficient; c i (n) is the Mel frequency cepstral coefficient corresponding to the n-th time frame.

[0128] After screening out the target Mel-frequency cepstral coefficients in this embodiment, a weighted feature is obtained by immediately using the trained neural network model. It should be noted that the above-mentioned target Mel-frequency cepstral coefficients are the MFCC feature maps. Please refer to Figure 8 , Figure 8 which shows the spectral information of the MFCC feature map.

[0129] In this embodiment, the trained neural network model includes a convolutional attention module. As a lightweight attention mechanism for convolutional neural networks, this convolutional attention module can adaptively enhance the important information in the feature map and improve the performance of the model. This convolutional attention module includes a channel attention module and a spatial attention module.

[0130] Please refer to Figure 9 , Figure 9 which shows the structural schematic diagram of the trained neural network model 100 in this embodiment. Among them, the channel attention module 102 and the spatial attention module 103 of the convolutional attention module 101 are connected in sequence. Please, on the basis of Figure 9 , refer to Figure 10 , Figure 10 which shows the step-by-step schematic diagram of step 300 in this embodiment; this step 300 includes step 301-step 302.

[0131] Step 301: Use the channel attention module to perform channel weighting on the target Mel-frequency cepstral coefficients to obtain an initial weighted feature;

[0132] Step 302: Use the spatial attention module to perform spatial weighting on the initial weighted feature to obtain the final weighted feature.

[0133] The following will introduce in detail the processing process of the convolutional attention module in this embodiment.

[0134] Among them, the channel attention module 102 is used to weight each channel to emphasize important feature channels. Specifically, global average pooling and global max pooling can be respectively performed on the input feature map, that is, the above-mentioned target Mel-frequency cepstral coefficients, to respectively obtain two channel descriptors, which are respectively expressed as:

[0135]

[0136] In the formula, GAP(F) is the channel descriptor corresponding to global average pooling; GMP(F) is the channel descriptor of global max pooling; H is the height corresponding to the target Mel-frequency cepstral coefficients; W is the width corresponding to the target Mel-frequency cepstral coefficients; C is the number of channels corresponding to the target Mel-frequency cepstral coefficients.

[0137] Further, the above two channel descriptors are respectively input into a multi-layer perceptron (MLP) to obtain corresponding attention weights. Subsequently, the sum of the attention weights corresponding to the above two channel descriptors is used as the channel attention feature corresponding to the channel attention module 102. Finally, the channel attention feature is multiplied by the target Mel-frequency cepstral coefficients channel by channel to obtain the initial weighted feature.

[0138] Among them, the initial weighted feature can be expressed as:

[0139]

[0140] Among them, F c is the initial weighted feature; M c is the channel attention feature; is matrix multiplication.

[0141] In this embodiment, the channel attention feature can be expressed as:

[0142] M c = σ(MLP(GAP(F)) + MLP(GMP(F)));

[0143] In the formula, σ(.) is the operator of Sigmoid(.); MLP(GAP(F)) is the channel attention weight corresponding to global average pooling; MLP(GMP(F)) is the channel attention weight of global max pooling.

[0144] Similarly, the spatial attention module 103 is used to perform spatial weighting on each spatial position of the above initial weighted feature, emphasizing important spatial positions to obtain the final weighted feature.

[0145] In this embodiment, the spatial attention module 103 respectively performs global averaging and global max pooling on the initial weighted feature output by the channel attention module 102 along the channel dimension to obtain two corresponding spatial descriptors. Subsequently, the two spatial descriptors are concatenated, and finally a convolutional layer is used to perform convolution on the concatenated spatial descriptors to obtain the corresponding spatial attention feature to enhance the feature information of important positions under the initial weighted feature.

[0146] In this embodiment, the spatial attention feature is expressed as:

[0147] M s = σ(Conv k×k ([GAP(F c )); GMP(F c )));

[0148] Among them, GAP(F c) is the spatial attention weight corresponding to global average pooling; GMP(F c ) is the spatial attention weight of global max pooling; Conv k×k (.) is the operator of the k×k convolutional layer.

[0149] Based on this, the final weighted feature F in this embodiment out satisfies:

[0150] To accelerate the processing speed of the neural network model and improve the accuracy, please continue to refer to Figure 9 In this embodiment, the trained neural network model 100 further includes a first convolutional layer 104, a first downsampling layer 105, a second convolutional layer 106, a second downsampling layer 107, and a fully connected layer 108.

[0151] Among them, the first convolutional layer 104, the first downsampling layer 105, the channel attention module 102, the spatial attention module 103, the second convolutional layer 106, the second downsampling layer 107, and the fully connected layer 108 are connected in sequence, and the output of the fully connected layer 108 is used as the output of the trained neural network model 100.

[0152] In a possible implementation manner, in this embodiment, the convolution kernel size of the first convolutional layer 104 is 5*5, which is used to extract the local features of the target Mel frequency cepstral coefficients and generate feature maps of multiple channels.

[0153] In this embodiment, the pooling window size of the first downsampling layer 105 can be 2*2, which is used to reduce the size of the feature map output by the first convolutional layer 104, reduce the computational amount of the neural network model, and at the same time enhance the invariance of the feature map.

[0154] In this embodiment, the convolution kernel size of the second convolutional layer 106 can be 5*5, and the second convolutional layer 106 is used to further extract features from the feature map output by the spatial attention module 103 and generate more high-level features.

[0155] In this embodiment, the pooling layer of the second downsampling layer 107 can be 2*2, which is used to further reduce the size of the feature map output by the second convolutional layer 106 and reduce the computational amount of the neural network model.

[0156] In a possible implementation manner, in this embodiment, the fully connected layer 108 under the neural network model is directly used to classify the weighted features, and the classification result is used as the prediction result of the electroencephalogram signal to be measured.

[0157] In this embodiment, the fully-connected layer 108 may include four sequentially-connected fully-connected layers 108. Each fully-sampled layer corresponds to different neurons. If they are the first fully-connected layer 108, the second fully-connected layer 108, the third fully-connected layer 108, and the fourth fully-connected layer 108 in sequence, the neurons corresponding to the first fully-connected layer 108, the second fully-connected layer 108, the third fully-connected layer 108, and the fourth fully-connected layer 108 may be 800, 120, 96, and 4 respectively. In this embodiment, the first fully-sampled layer is used to flatten the feature map output by the second down-sampling layer 107 and perform feature combination. The second fully-connected layer 108 is used to further extract the high-level features output by the first fully-sampled layer and reduce the dimension. The third fully-connected layer 108 is used to output the final weighted features. The fourth fully-connected layer 108 classifies the final weighted features, where the 4 neurons may correspond to 4 categories.

[0158] In this embodiment, taking the electroencephalogram (EEG) signal as an example, multiple groups of sample data may be signals at different stages of the theoretical epileptic EEG signal. For example, the theoretical EEG signals corresponding to Pre1 before seizure, Pre2 during seizure, Seizure after seizure, and Inter-ictal during the seizure period. After obtaining the weighted features of the EEG signal to be measured, the weighted features can be labeled according to the above-mentioned theoretical EEG signals corresponding to Pre1 before seizure, Pre2 during seizure, Seizure after seizure, and Inter-ictal during the seizure period, and the labeling result can be used as the prediction result of the EEG signal to be measured.

[0159] Based on this, in this embodiment, the neural network model uses multi-layer convolution and pooling operations to gradually extract features, combines the above-mentioned convolutional attention module to enhance the discrimination ability of important features, and finally performs classification output through the fully-connected layer.

[0160] In summary, the present invention fuses the Mel-frequency cepstral coefficients (MFCCs) with the attention mechanism, uses the MFCCs to emphasize the spectral features of low frequencies, enables the neural network based on the attention mechanism to better identify the information of low-frequency spectra, thereby improving the prediction accuracy. At the same time, the logarithmic operation and discrete cosine transform of the MFCCs greatly reduce the dimension of the features, further improving the recognition speed of the deep neural network.

[0161] In a second aspect, an embodiment of the present invention further provides an electronic device, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor. The processor can execute the machine-executable instructions to implement the method for predicting electroencephalogram signals based on a neural network described in any one of the above first aspects. For example: acquiring and converting an electroencephalogram signal to be measured into a power spectrum image, and after obtaining the target Mel frequency cepstral coefficients of the power spectrum image, inputting the target Mel frequency cepstral coefficients into a trained neural network model 100 to obtain weighted features through the attention mechanism in the neural network model. Finally, classifying the weighted features using multiple sets of sample data, and using the classification result as the prediction result of the electroencephalogram signal to be measured.

[0162] In this embodiment, the memory is used to store programs or data. The memory may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0163] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium. The computer-readable storage medium stores computer instructions and programs, and when the computer instructions and programs are read and run, they execute the methods of the above embodiments. The storage medium may include memory, flash memory, registers, or a combination thereof, etc.

[0164] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for predicting electroencephalogram signals based on a neural network, characterized in that: The following steps are involved: Acquire an electroencephalogram signal to be tested, and convert the electroencephalogram signal to be tested into a power spectrum image; Obtain target Mel-frequency cepstrum coefficients of the power spectrum image; Inputting the target Mel frequency cepstral coefficient into a trained neural network model to obtain weighted features; wherein the trained neural network model includes a convolutional attention module; The weighted features are classified using multiple groups of sample data, and the classification results are used as the prediction results of the electroencephalogram signal to be tested.

2. The method for predicting an electroencephalogram signal according to claim 1, characterized in that: The step of obtaining target Mel-frequency cepstrum coefficients of the power spectrum image comprises: The power spectrum image is converted into a Mel spectrum using a Mel filter bank; Performing discrete cosine transform on the Mel frequency spectrum to obtain a plurality of Mel frequency cepstrum coefficients; A target Mel-frequency cepstral coefficient having an order smaller than a set value is screened out from the multiple Mel-frequency cepstral coefficients.

3. The method for predicting an electroencephalogram signal according to claim 1 or 2, characterized in that: The target Mel frequency cepstral coefficient is expressed as: MFCC i =[c i (1),c i (2),…,c i (L)]; Among them, MFCC i is the target Mel frequency cepstrum coefficient; c i (n) is the Mel-frequency cepstral coefficient corresponding to the nth time frame; M is the number of filters in the Mel filter bank; log(.) is the logarithmic function; L is the number of target Mel-frequency cepstral coefficients; cos(.) is the cosine function.

4. The method for predicting an electroencephalogram signal according to claim 1 or 2, characterized in that: The convolutional attention module includes a channel attention module and a spatial attention module; The step of inputting the target Mel frequency cepstral coefficient into the trained neural network model to obtain weighted features includes: Using the channel attention module to perform channel weighting on the target Mel-frequency cepstral coefficients to obtain initial weighted features; The spatial attention module is used to perform spatial weighting on the initial weighted features to obtain final weighted features.

5. The method for predicting electroencephalogram signals according to claim 4, characterized in that: The weighted feature is expressed as: Among them, F out is the weighted feature; g c is the initial weighted feature; M c is the channel attention feature; M S is the spatial attention feature; is matrix multiplication.

6. The method for predicting electroencephalogram signals according to claim 4, characterized in that: The trained neural network model also includes a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer and a fully connected layer; Among them, the first convolutional layer, the first downsampling layer, the channel attention module, the spatial attention module, the second convolutional layer, the second downsampling layer and the fully connected layer are connected in sequence, and the output of the fully connected layer is used as the output of the trained neural network model.

7. The method for predicting electroencephalogram signals according to claim 1 or 2, characterized in that: Before the step of converting the EEG signal to be tested into a power spectrum image, the following steps are also included: The electroencephalogram signal to be measured is preprocessed to filter out high-frequency components in the electroencephalogram signal to be measured.

8. The method for predicting electroencephalogram signals according to claim 2, characterized in that: The design parameters of the Mel filter bank satisfy: Among them, mel(.) is the Mel frequency; f is the linear frequency; log(.) is the logarithmic function.

9. The method for predicting electroencephalogram signals according to claim 8, characterized in that: The Mel spectrum satisfies: Among them, S i (m) is the Mel frequency corresponding to the mth filter in the Mel filter bank; M is the number of filters in the Mel filter bank; P i (k) is the power spectrum; H m (k) is the frequency response corresponding to the mth filter; N is the power spectrum P i (k) is the total number of frequency points k.

10. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the neural network-based EEG signal prediction method described in any one of claims 1-9.

Images