A method for automatic generation of high-frequency oscillation labels based on semi-supervised learning

Abstract

The present invention discloses a method for automatically generating high-frequency oscillation labels based on semi-supervised learning to alleviate the problem of severe lack of HFOs labels. First, in order to focus on both temporal and spatial features, we designed a network architecture consisting of a recurrent neural network and a convolutional neural network. Secondly, in order to make full use of a large amount of unlabeled data, we explored a semi-supervised algorithm consisting of a supervised component and an unsupervised component. The supervised component minimizes the cross entropy of the input and output labels using labeled samples. At the same time, the unsupervised component maximizes the consistency of the output based on the original input and the perturbed input, and uses a large amount of unlabeled data samples to generate HFOs labels, thereby improving the utilization of unlabeled data to reduce the cost of labels while improving the accuracy and speed of HFOs labeling.

Description

Technical Field

[0001] The present invention relates to the technical field of signal detection, and in particular to a method for automatically generating high-frequency oscillation labels based on semi-supervised learning. Background Art

[0002] Epilepsy accounts for a large proportion of the world's disease burden, affecting approximately 50 million people worldwide. Among patients with epilepsy, one-third do not respond to drug treatment (patients with refractory epilepsy). Therefore, patients with refractory epilepsy may be seizure-free through epilepsy surgery, but the effectiveness of epilepsy surgery still depends on accurate localization of the epileptogenic zone (EZ).

[0003] Accurate localization of epileptogenic lesions is an important prerequisite for improving the prognosis of drug-resistant epilepsy. High-frequency oscillations (HFOs) in intracranial electroencephalogram (iEEG) recordings are considered to be new clinical biomarkers for effective localization of epileptogenic lesions and are crucial for accurate localization of the EZ. There is a significant correlation between the resection of HFO-producing tissue and a good prognosis. According to frequency, HFOs are divided into two types: Rs (Ripples, 80-250Hz) and FRs (Fast Ripples, 250-500Hz).

[0004] Currently, visual markers of HFOs based on intracranial electroencephalography (iEEG) and video recordings are considered the diagnostic standard for clinicians. However, EEG recordings usually require continuous monitoring for several days, and HFO signals are short in duration and small in amplitude. Therefore, it is difficult for clinicians to manually analyze such a large amount of HFOs data. Therefore, it is of great significance to find an automatic detection method for HFOs.

[0005] In previous studies, many automatic detection methods for HFOs have been proposed. Based on the morphological characteristics of HFOs, researchers have used a variety of methods to detect HFOs, such as short-term line length features, Hilbert transform envelope, and composite wavelet transform. In recent years, more and more machine learning methods have been applied to detect HFOs. Jrad et al. used Gabor transform and used a multi-classification SVM; Wan et al. used fuzzy entropy and short-term energy as input and used fuzzy neural network for HFO detection; Burelo et al. proposed a spike neural network (SNN) method.

[0006] However, these automatic detection methods usually require sufficient labels for effective training. In practice, labeling medical data requires the expertise of experienced clinicians and may take a lot of time. In addition, visual analysis is affected by various subjective and objective factors of doctors, which inevitably leads to omissions or wrong labels. The number of labeled data samples is small, which seriously affects the results of the computational model. In contrast, unlabeled data is easy to obtain and the amount of data is huge. Therefore, it is necessary to consider how to use a large number of unlabeled data samples. Summary of the invention

[0007] In view of this, the present invention optimizes and innovates the current automatic detection method of HFOs, and proposes a method for automatically generating high-frequency oscillation labels based on semi-supervised learning.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A method for automatically generating high-frequency oscillation labels based on semi-supervised learning, which uses a neural network including a recurrent neural network and a convolutional neural network to extract the temporal and spatial features of HFOs signals, then fuses the temporal and spatial features to obtain a feature vector of HFOs, and finally uses a multilayer perceptron to identify the results; the neural network uses a consistency regularized semi-supervised algorithm to optimize model parameters.

[0010] Furthermore, the semi-supervised algorithm of consistency regularization includes the following steps:

[0011] S1. Initialize two network models with the same structure; each network model includes a recurrent neural network and a convolutional neural network. The recurrent neural network is used to extract the temporal features of the HFOs signal, and the convolutional neural network is used to extract the spatial features of the HFOs signal;

[0012] S2, adding different Gaussian white noises to the input filtered signal, taking them as the input of the network model, and inputting them into two network models with the same structure;

[0013] S3. Train the model parameters of one of the network models according to the loss function. The loss function consists of two parts: unsupervised loss and supervised loss. The unsupervised loss is used to minimize the difference between the different outputs of the two networks corresponding to the same input of the labeled data and the unlabeled data. The supervised loss is only used to calculate the labeled data. The weight of the unsupervised loss gradually increases during the training process.

[0014] S4. Perform exponential moving average on the model parameters trained in step S3 to obtain model parameters of another model.

[0015] Furthermore, the unsupervised loss uses MSE loss.

[0016] Furthermore, the supervised loss uses cross entropy loss.

[0017] Furthermore, the recurrent neural network is composed of LSTM and is used to learn the temporal characteristics of the signal.

[0018] Furthermore, the number of hidden units in the LSTM network is 100 and the number of layers is 2.

[0019] Furthermore, the dimension of the RNN output is 1*100.

[0020] Furthermore, the convolutional neural network is rewritten into a one-dimensional version using the core idea of ​​the residual neural network to learn the spatial characteristics of the signal.

[0021] Furthermore, the dimension of the convolutional neural network output is 1*512.

[0022] Furthermore, the number of hidden units of the multilayer perceptron is 300.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] Considering the difficulty and high cost of obtaining high-quality HFOs labels, the method for automatically generating high-frequency oscillation labels based on semi-supervised learning provided by the present invention is used to alleviate the serious lack of HFOs labels. First, in order to focus on both temporal and spatial features, we designed a network architecture consisting of a recurrent neural network and a convolutional neural network. Secondly, in order to make full use of a large amount of unlabeled data, we explored a semi-supervised algorithm consisting of supervised and unsupervised components. The supervised component minimizes the cross entropy of the input and output labels using labeled samples. At the same time, the unsupervised component maximizes the consistency of the output based on the original input and the perturbed input, and uses a large number of unlabeled data samples to generate HFOs labels, thereby improving the utilization of unlabeled data to reduce the cost of labels while improving the accuracy and speed of HFOs labeling. Experiments have shown that our method achieves high accuracy and sensitivity when only 5% of labeled data is used. In addition, in terms of clinical application, we divide the data by patient, avoiding the problem of data leakage in existing studies. In the case of using clinical data, our method effectively alleviates the lack of HFOs labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0026] Figure 1 A method for automatically generating high-frequency oscillation labels based on semi-supervised learning is provided in an embodiment of the present invention.

[0027] Figure 2 A preprocessing flow chart provided for an embodiment of the present invention.

[0028] Figure 3 The confusion matrix diagram of the comparative test provided by the embodiment of the present invention. In the figure, (a) is Pseudo-Label, (b) is Pi-Model, (c) is Mean-Teacher, and (d) is the method of the present invention. NPV: negative predictive value; Prec: precision; Acc: accuracy; Spec: specificity; Sens: sensitivity. DETAILED DESCRIPTION

[0029] Although many automatic detection methods for HFOs have been proposed, labeling HFOs is an expensive, subjective, time-consuming and laborious task. In the absence of HFOs labels, the existing automatic detection methods have poor performance and serious misdiagnosis and missed diagnosis problems. Based on this, we optimize and innovate the current automatic detection methods for HFOs and propose a method for automatic generation of high-frequency oscillation labels based on semi-supervised learning (CRM). First, our network architecture consists of two parts: a recurrent neural network (RNN) and a convolutional neural network (CNN), which consider the temporal and spatial characteristics of HFOs signals respectively. Finally, the features of the two angles are fused to obtain the feature vector of HFOs. Secondly, in order to utilize a large amount of unlabeled data, we explored a semi-supervised algorithm consisting of supervised and unsupervised parts, using different loss function calculation methods for labeled and unlabeled samples.

[0030] In order to better understand the technical solution, the method of the present invention is described in detail below with reference to the accompanying drawings.

[0031] The overall process of the method proposed by the present invention is as follows Figure 1As shown, first, the original EEG signal is preprocessed. Then, in order to focus on the temporal and spatial characteristics of HFOs at the same time, the neural network architecture of the present invention includes two parts: a recurrent neural network and a convolutional neural network. The recurrent neural network is composed of LSTM, and the convolutional neural network part uses the core idea of ​​the residual neural network (ResNet) to characterize the filtered signal data, and the outputs of the two parts are feature-fused through a fusion module. Then, a multi-layer perceptron (MLP) classifier is used to identify the results. Finally, in order to alleviate the problem of severe lack of labeled data, the present invention designs a semi-supervised algorithm based on consistency regularization, and uses labeled data and unlabeled data to calculate the loss function to optimize the model parameters.

[0032] The overall process of preprocessing is as follows Figure 2 As shown, the following steps are included:

[0033] Step 1: The waveform and amplitude of the bipolar lead are less distorted, so the original EEG needs to be polarity-converted;

[0034] Step 2: Remove the channels and empty electrodes that are obviously interfered with;

[0035] Step 3: Filter out the power frequency interference and its multiplier interference through a 50Hz multiplier notch filter on the dual-conductor signal;

[0036] Step 4: Use a bandpass filter to retain the EEG signal in the frequency range of 80-500 Hz.

[0037] The neural network architecture used in the automatic generation method of high-frequency oscillation labels based on semi-supervised learning is as follows: Figure 1 As shown in the figure, the network architecture is divided into two parts: the recurrent network and the convolutional network.

[0038] In the recurrent network part, a long short-term memory (LSTM) neural network is introduced to learn the temporal correlation of the signal, considering the temporal characteristics of the input signal. Specifically, the number of hidden units and the number of layers of the LSTM network used are 100.

[0039] The convolutional network part pays more attention to the spatial characteristics of the input signal and rewrites it into a one-dimensional version based on the core idea of ​​the residual neural network (Resnet). Specifically, the input signal first passes through a convolutional layer with a kernel size of 15. Then, the output of the convolutional network part is obtained through four residual blocks, each of which consists of two convolutional layers with a kernel size of 7. The number of block stacks is [2, 2, 2, 2].

[0040] Finally, the output of the recurrent network part and the output of the convolutional network part are feature fused. The dimension of the recurrent neural network output is 1*100, and the dimension of the convolutional neural network output is 1*512, so the dimension after fusion is 1*612. The final output is obtained through a multi-layer perceptron (MLP) with 300 hidden units.

[0041] The specific process of the semi-supervised algorithm is as follows:

[0042] Step 1: Initialize two models with the same structure;

[0043] Step 2: Add different Gaussian white noises to the input filtered signal, use them as the input of the model, and input them into two models with the same structure;

[0044] Step 3: Train the model parameters of one of the models according to the loss function.

[0045] We denote all training data as D and labeled data as D l , the unlabeled data is represented as D ul .

[0046] Our loss function consists of two parts: unsupervised loss (L u ) and supervised loss (L s ).

[0047] In the unsupervised loss part, we use MSE loss to minimize D l and D ul The difference between the two networks' different outputs for the same input is shown below:

[0048]

[0049] Among them, f θ (·) and f θ′ (·) are two networks with parameters θ and θ′, h(·) is a random noise input function, and B is the batch size of D.

[0050] In the supervised loss part, we only use the cross entropy loss L on the labeled data s as follows:

[0051]

[0052] Due to the randomness of network initialization, at the beginning of training, we hope that the weight of the supervised loss term is relatively large. As training progresses, we hope to slowly increase the weight of the unsupervised loss term to speed up the convergence of the model. Therefore, we use a weight increase function. In the unsupervised loss term, the weight function It increases with the increase of training period t.

[0053]

[0054] Therefore, the total loss is calculated as follows:

[0055]

[0056] Step 4: Perform exponential moving average (EMA) on the parameters of the above trained model to obtain the parameters of another model. The specific formula is as follows:

[0057] θ′ t =αθ′ t-1 +(1-α)θ t

[0058] like Figure 3 As shown in Figure 2, experiments show that our method achieves high accuracy and sensitivity using only 5% labeled data.

[0059] In addition, in early studies, researchers mostly focused only on the performance of signal detection, so in terms of experimental data division, they mostly adopted a random division method for training sets and test sets. This method can check the performance of the model to a certain extent. However, there may be different data of the same patient in the training set and the test set, so this method introduces data leakage problems to a certain extent. In this case, when the model is applied to new patients, it is likely that the performance will drop significantly and fail to meet clinical needs.

[0060] In actual clinical applications, ideally, when considering a new patient, the prior knowledge gained from existing cases needs to be transferred to the judgment of the new patient. Therefore, the generalization ability of the model between different patients must be considered.

[0061] We evaluated the method of the present invention on a SEEG-based high-frequency oscillation dataset and a private clinical dataset. In our method, considering the above clinical needs, the leave-one-out method was used for cross-validation and the data was divided based on patients to ensure that the training set and the test set did not have data from the same patient.

[0062] The results show that our method can accurately label HFOs data when there are fewer labels. As shown in Table 1. Even in cross-validation, our detector still performs well, and all indicators are better than similar studies. As shown in Table 2. This has proved that it is feasible to improve the accuracy of HFOs labeling using relatively easy-to-obtain unlabeled data, and our method can be applied to clinical applications, alleviating the serious lack of HFOs labeling and reducing labeling costs.

[0063] Table 1 Experimental results of clinical verification

[0064]

[0065] Table 2 Performance comparison of comparative tests

[0066]

[0067] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically generating high-frequency oscillation labels based on semi-supervised learning, It is characterized in that A neural network including a recurrent neural network and a convolutional neural network is used to extract the temporal and spatial features of the HFOs signal, and then the temporal and spatial features are fused to obtain the feature vector of the HFOs, and finally a multilayer perceptron is used to identify the result; the neural network uses a consistency regularized semi-supervised algorithm to optimize the model parameters; the consistency regularized semi-supervised algorithm includes the following steps: S1. Initialize two network models with the same structure; each network model includes a recurrent neural network and a convolutional neural network. The recurrent neural network is used to extract the temporal features of the HFOs signal, and the convolutional neural network is used to extract the spatial features of the HFOs signal; S2, adding different Gaussian white noises to the input filtered signal, taking them as the input of the network model, and inputting them into two network models with the same structure; S3. Train the model parameters of one of the network models according to the loss function. The loss function consists of two parts: unsupervised loss and supervised loss. The unsupervised loss is used to minimize the difference between the different outputs of the two networks corresponding to the same input of labeled data and unlabeled data. The supervised loss is only used to calculate the labeled data. The weight of the unsupervised loss gradually increases during the training process. The total loss function is calculated as follows: Among them, L u is the unsupervised loss, L s is the supervised loss, is the weight of the unsupervised loss, the weight function It is expressed as: Where t is the training period; S4. Perform exponential moving average on the model parameters trained in step S3 to obtain model parameters of another model.

2. The method for automatically generating high-frequency oscillation labels based on semi-supervised learning according to claim 1, It is characterized in that The unsupervised loss uses MSE loss.

3. The method for automatically generating high-frequency oscillation labels based on semi-supervised learning according to claim 1, It is characterized in that The supervised loss described uses cross entropy loss.

4. The method for automatically generating high-frequency oscillation labels based on semi-supervised learning according to claim 1, It is characterized in that The recurrent neural network is composed of LSTM and is used to learn the temporal characteristics of signals.

5. The method for automatically generating high-frequency oscillation labels based on semi-supervised learning according to claim 4, It is characterized in that The LSTM network has 100 hidden units and 2 layers.

6. The method for automatically generating high-frequency oscillation labels based on semi-supervised learning according to claim 5, It is characterized in that The dimension of the recurrent neural network output is 1*100.

7. The method for automatically generating high-frequency oscillation labels based on semi-supervised learning according to claim 1, It is characterized in that The convolutional neural network is rewritten as a one-dimensional version using the core idea of ​​the residual neural network to learn the spatial characteristics of the signal.

8. The method for automatically generating high-frequency oscillation labels based on semi-supervised learning according to claim 7, It is characterized in that The dimension of the convolutional neural network output is 1*512.

9. The method for automatically generating high-frequency oscillation labels based on semi-supervised learning according to claim 1, It is characterized in that The number of hidden units of the multilayer perceptron is 300.

Images