Double-branch microevent detection method based on weak supervision

Abstract

The invention discloses a weak supervision-based double-branch micro-event detection method, which comprises the following steps of: acquiring a sleep electroencephalogram signal, constructing a data set and dividing the data set into a training set, a test set and a verification set; preprocessing the data in the data set; constructing a double-branch micro-event detection network model based on weak supervision, wherein the model comprises a feature extraction network, a branch network I, a clustering module and a branch network II; training the network model by using a data set, and adjusting network parameters by using an optimization method; and inputting the electroencephalogram signal into the trained network model, and detecting and extracting a micro event. According to the double-branch micro-event detection method based on weak supervision, the sleep stage coarse-grained label easy to obtain is used for training, and the problem of scarcity of high-quality micro-event labeling data is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electroencephalogram signal processing, and particularly relates to a weakly supervised dual-branch micro-event detection method. Background Art

[0002] Research shows that there is a significant association between sleep micro-events and human neurocognitive functions, which has attracted extensive attention from academic and clinical researchers. For example, research shows that sleep spindles are closely related to cognitive functions such as human memory consolidation and learning ability, while K-complex waves are importantly related to neural activities such as the brain's self-regulation and perceptual threshold. Therefore, the extraction and analysis of sleep micro-events, especially features such as sleep spindles and K-complex waves, have become important research directions in the fields of neuroscience, sleep medicine, and cognitive psychology.

[0003] However, in a hospital environment, the extraction of sleep micro-events mainly relies on manual annotation by experts, which is not only time-consuming and laborious, but may also affect the accuracy of the results due to subjective differences among experts. Therefore, the realization of automatic annotation of sleep micro-events has become an important area of sleep research. With the development of technologies such as deep learning, automatic annotation not only improves efficiency, but also reduces subjective errors and enhances the consistency and accuracy of annotation, attracting the attention of more and more experts.

[0004] However, there are still the following problems in using deep learning methods for sleep micro-event extraction at present: 1) As a data-driven technology, deep learning usually requires a large amount of labeled data for training, but there is a lack of sufficient micro-event labeled data in the hospital scenario; 2) The existing micro-event annotation is greatly affected by expert subjectivity, resulting in significant inconsistencies among experts, thus affecting the quality of data labels and further limiting the application effect of deep learning methods in micro-event extraction; 3) The existing micro-event extraction methods can only identify single-category micro-events, and different recognition strategies need to be adopted for different types of sleep micro-events, which not only increases the complexity of method design, but also makes its practical application in clinics more difficult. Summary of the Invention

[0005] The present invention provides a weakly supervised dual-branch micro-event detection method to solve the above-mentioned technical problems, and specifically adopts the following technical solutions:

[0006] A weakly supervised dual-branch micro-event detection method includes the following steps:

[0007] Obtain sleep electroencephalogram signals, construct a data set and divide it into a training set, a test set, and a validation set;

[0008] Preprocess the data in the data set;

[0009] Construct a dual-branch micro-event detection network model based on weak supervision. The model includes a feature extraction network, a first branch network, a clustering module, and a second branch network;

[0010] Use the dataset to train the network model and adjust the network parameters using an optimization method;

[0011] Input the electroencephalogram (EEG) signal into the trained network model to detect and extract micro-events.

[0012] Further, the specific method for obtaining the sleep EEG signal, constructing the dataset, and dividing it into a training set, a test set, and a validation set is as follows:

[0013] Collect the sleep EEG signal and divide it into a series of data segments with a duration of 30 seconds;

[0014] Divide the data into a training set, a validation set, and a test set at a ratio of 8:1:1;

[0015] Perform coarse-grained annotation on all data segments, classify each segment according to the sleep staging standard, which are the wakefulness period, N1 stage, N2 stage, N3 stage, and REM stage respectively. Perform fine-grained annotation on the data segments in the test set, specifically annotate the occurrence intervals of sleep micro-events, and construct a sleep EEG signal database containing coarse-grained sleep staging and fine-grained micro-event annotations.

[0016] Further, the steps for obtaining the sleep EEG signal data and performing preprocessing include:

[0017] Denoise the EEG signal to remove artifacts such as electrooculogram (EOG) and electromyogram (EMG).

[0018] Further, the feature extraction network includes:

[0019] A candidate region generation module that uses a sliding window method to generate candidate signal segments with a certain overlap from the input signal as candidate regions;

[0020] A feature extraction module that, based on the DarkNet53 network structure of YOLOv3, extracts deep features through multiple convolutional layers and residual modules and inputs them into the first branch network and the second branch network.

[0021] Further, the first branch network converts the Euclidean distance between the deep features and the waveform prototype vectors into a similarity score, generates a similarity score matrix and inputs it into the clustering module, and after summing the similarity scores of all candidate segments, uses a fully connected network to obtain the sleep staging prediction.

[0022] Further, the clustering module calculates the clustering clusters through the similarity score matrix.

[0023] Further, the specific method for the clustering module to calculate the clustering clusters through the similarity score matrix is as follows:

[0024] The clustering module performs clustering on the similarity score matrix in the candidate region dimension to find the clustering center;

[0025] Based on the clustering center, the candidate region clusters of micro-events existing in the input signal are obtained.

[0026] Further, the second branch network converts the Euclidean distance between the deep feature and the waveform prototype vector into a similarity score to generate a similarity score matrix. The second branch network and the first branch network share the same waveform prototype vector;

[0027] The second branch network calculates the detection pseudo-labels and label weights of micro-events according to the clustering clusters, and trains the network by calculating the loss function through the similarity score matrix and the pseudo-labels.

[0028] Further, the training of the network model adopts the Adam optimization method to dynamically adjust the learning rate.

[0029] Further, the training of the network model adopts a composite loss function L, which is defined as follows:

[0030]

[0031] where are real number parameters, which are used to adjust the ratio between each function respectively, and are defined as follows:

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038] where, is the number of sleep stage types, is the label of the electroencephalogram signal sleep stage, represents that the model predicts the sleep stage as probability, is the prototype feature learned by the network, is the deep feature vector extracted in the feature extraction network, is the number of input EEG signals, is the number of prototype features in the network, is the weight parameter of the fully connected layer in the first branch network, is the pseudo-label weight.

[0039] The beneficial effect of the present invention lies in the provided dual-branch micro-event detection method based on weak supervision, which uses easily obtained coarse-grained labels of sleep staging for training, solves the problem of scarce high-quality micro-event annotation data, and reduces the impact of expert subjectivity on the quality of data labels through the weak supervision learning method. It provides a dual-branch network structure capable of detecting multiple micro-events simultaneously, simplifies the method design, improves the detection efficiency and accuracy, provides important technical support for neuroscience research and clinical diagnosis, and helps to deeply understand the mechanisms of cognitive functions and neurological diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 is the overall architecture of the dual-branch micro-event detection network based on weak supervision of the present application;

[0042] Figure 2 is the structural diagram of the feature extraction module of the present application;

[0043] Figure 3 is the flow chart of the clustering module of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] The following details the embodiments of the present application. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application, but should not be construed as limiting the present application.

[0045] The present application discloses a dual-branch micro-event detection method based on weak supervision, including the following steps:

[0046] S1: Obtain sleep EEG signals, construct a data set and divide it into a training set, a test set, and a validation set.

[0047] In the implementation manner of the present application, the specific method for obtaining sleep EEG signals, constructing a data set and dividing it into a training set, a test set, and a validation set is:

[0048] First, collect sleep EEG signal data with a sampling rate of 100 Hz and divide it into a series of data segments with a duration of 30 seconds. Subsequently, divide the data into a training set, a validation set, and a test set, with proportions of 80%, 10%, and 10% respectively, and there is no overlap among the three. Then, clinical experts perform coarse-grained annotation on these 30-second data segments, that is, classify each segment according to the sleep staging standard, which are wakefulness period, N1 stage, N2 stage, N3 stage, and REM stage. In addition, clinical experts also perform fine-grained annotation on the data segments in the test set, specifically annotating the occurrence intervals of sleep micro-events, including characteristic events such as spindle waves and K-complex waves. Finally, a sleep EEG signal database containing coarse-grained sleep staging and fine-grained micro-event annotation is constructed. The number of sampling points of the signal samples in this dataset is 3000 (30s )

[0049] S2: Preprocess the data in the dataset.

[0050] Specifically, perform denoising processing on the EEG data to remove electrooculogram noise and electromyogram noise in it.

[0051] S3: Construct a weakly supervised dual-branch micro-event detection network model.

[0052] In the implementation manner of this application, the network model includes a feature extraction network, a branch network one, a clustering module, and a branch network two.

[0053] In the implementation manner of this application, the feature extraction network includes: a candidate region generation module and a feature extraction module.

[0054] The candidate region generation module uses a sliding window method to generate candidate signal segments with a certain overlapping part from the input signal as candidate regions. Specifically, given an input signal with a length of T, this module slides step by step on the signal in a way of a fixed window size of length L and a step size of S to segment the signal, forming R time segments with a certain overlapping part , where refers to the i-th candidate signal segment.

[0055] The feature extraction module is based on the DarkNet53 network structure of YOLOv3 and extracts deep features through multiple convolutional layers and residual modules that can be input into branch network one and branch network two. Specifically, the feature extraction module draws on the DarkNet53 network structure of YOLOv3. As Figure 2 ​As shown. This network contains a convolutional layer at each of the input and output layers, and the middle layer is composed of four convolutional layers, four residual modules, and a downsampling layer connected in series. Each residual module is composed of multiple repeatedly stacked convolutional layers and residual connections, where the parameters of the one-dimensional convolutional layer are (16, 3, 1), indicating that the number of convolutional channels is 16, the convolutional kernel size is 3, and the stride is 1.

[0056] In an embodiment of the present application, Branch Network 1 converts the Euclidean distance between the deep features and the waveform prototype vectors into a similarity score, generates a similarity score matrix and inputs it into the clustering module, and after summing the similarity scores of all candidate segments, uses a fully connected network to obtain the sleep stage prediction.

[0057] Specifically, Branch Network 1 is a network with sleep stages as labels, which embeds M learnable waveform prototype vectors . The deep features learned by the feature extraction network can be regarded as a set of R signal features . This network calculates the Euclidean distance between each waveform prototype and the signal features, converts it into a similarity score, and generates a similarity score matrix , where represents the similarity score between the i-th waveform prototype and the j-th candidate signal segment. After obtaining the similarity score matrix of each candidate segment , it is converted into the similarity score of the entire signal by summing all regions. Subsequently, a fully connected layer is used to assign weights to the similarity scores, and the output result is processed by the Softmax function to obtain the predicted sleep stage .

[0058] The clustering module calculates the clustering clusters through the similarity score matrix. Specifically, the specific method for the clustering module to calculate the clustering clusters through the similarity score matrix is: the clustering module clusters the similarity score matrix in the candidate region dimension and finds the clustering centers. According to the clustering centers, the candidate region clustering clusters of the micro-events existing in the input signal are obtained.

[0059] Specifically, the clustering module calculates the clustering clusters through the similarity score matrix . This process is divided into two steps: the first step is to cluster the similarity score matrix in the candidate region dimension and find the clustering centers; the second step is to obtain the candidate region clustering clusters of the micro-events existing in the input signal according to the clustering centers.

[0060] The whole process is as Figure 3 shown. In the first step, the element with the largest score is found from the similarity score matrix , and the corresponding waveform prototype , where the calculation process of k is as follows:

[0061]

[0062] The waveform prototype mainly exists in the candidate signal segment corresponding to this element . Then, calculate the similarity scores of R candidate regions and . Subsequently, use the KMeans clustering algorithm to cluster the similarity scores (the number of clusters k = 5), and select the cluster with the maximum similarity score among them . The candidate signal segment corresponding to this cluster may contain the waveform prototype . Then, arrange in the original order in , group the adjacent similarity scores, and define the similarity score with the largest value in each group as the clustering center , and its index set is , where n represents the number of clustering centers. The candidate segments represented by the clustering centers, that is, , represent the core parts of multiple waveform prototypes existing in the input signal. So far, the network has completed clustering of the candidate regions and found the clustering centers. In the second step, for each clustering center, take elements with a distance of on its left and right respectively to jointly form clustering clusters with different clustering centers. Each clustering cluster contains the sleep micro-events corresponding to the waveform prototype , and its expression is as follows:

[0063]

[0064] In the implementation manner of the present application, Branch Network 2 converts the Euclidean distance between the deep features and the waveform prototype vectors into similarity scores, generates a similarity score matrix, and Branch Network 2 and Branch Network 1 share the same waveform prototype vectors. Then, Branch Network 2 calculates the pseudo-labels and label weights of the micro-events according to the clustering clusters, and trains the network by calculating the loss function through the similarity score matrix and the pseudo-labels.

[0065] Specifically, Branch Network 2 processes the deep features s learned by the feature extraction network through the same process as Branch Network 1 to obtain the similarity score matrix . Among them, Branch Network 2 and Branch Network 1 share the same waveform prototype vectors. For the clustering cluster input to Branch Network 2, first find its index position in the similarity score matrix to obtain the index matrix , and its expression is as follows:

[0066] According to the index matrix , the pseudo-labels for micro-event extraction can be obtained . Set the labels at the corresponding positions in the index matrix to 1, and the labels at the remaining positions to 0. The specific formula is as follows:

[0067]

[0068] In this process, the candidate segments included in this clustering cluster are labeled as the sleep micro-events corresponding to the waveform prototype . Subsequently, according to the similarity scoring matrix and the pseudo-labels calculate the loss function to train the network.

[0069] S4: Use the dataset to train the network model and adopt an optimization method to adjust the network parameters.

[0070] In the embodiment of the present application, the Adam optimization method is adopted for training the network model to dynamically adjust the learning rate.

[0071] Specifically, the number of samples used for each model training is 32, and the samples are traversed 1000 times during training. If the optimal result has not been updated within 50 times, the training is terminated in advance, and the learning rate for training is 5e-4. The training of the network model adopts a composite loss function L, which is defined as follows:

[0072]

[0073] Among them are real number parameters, which are used to adjust the ratio between each function respectively, and are defined as follows:

[0074]

[0075]

[0076]

[0077]

[0078]

[0079]

[0080] Among them, is the number of types of sleep stages, is the label of the electroencephalogram signal sleep stage, represents the probability that the model predicts the sleep stage as , is the prototype feature learned by the network, is the deep feature vector extracted in the feature extraction network, is the number of input EEG signals, is the number of prototype features in the network, is the weight parameter of the fully connected layer in the first branch network, is the pseudo-label weight.

[0081] S5: Input the EEG signal into the trained network model to detect and extract micro-events.

[0082] In the implementation mode of the present application, given the EEG signal , input it into the trained micro-event detection network model to obtain R candidate signal segments and the similarity score matrix . Find the similarity scores of the R candidate regions in the similarity score matrix with . Then, referring to the process in the clustering module, cluster by the similarity score to obtain the spindle wave clustering cluster and its index matrix . This spindle wave clustering cluster is considered to be the set of scores with the highest similarity to the spindle wave, and the candidate signal segments corresponding to its index matrix are the spindle wave candidate signals . After the model detection, the research will screen the detection results in combination with the known frequency characteristics of micro-events. Taking spindle waves as an example, their common frequency range is 12 - 14 Hz. By calculating the relative power ratio of the spindle wave candidate signal in this frequency band and setting a threshold , only when the relative power ratio of the candidate signal reaches this threshold is it regarded as a valid spindle wave event.

[0083] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art of this industry should understand that the above embodiments do not limit the present invention in any form. Any technical solutions obtained by means of equivalent replacement or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A weakly supervised dual-branch micro-event detection method, characterized in that, It includes the following steps: Obtain sleep EEG signals, construct a dataset and divide it into a training set, a test set, and a validation set; Preprocess the data in the dataset; Construct a weakly supervised dual-branch micro-event detection network model, which includes a feature extraction network, a first branch network, a clustering module, and a second branch network; Use the dataset to train the network model and adjust the network parameters using an optimization method; Input the EEG signals into the trained network model to detect and extract micro-events.

2. The weakly supervised dual-branch micro-event detection method according to claim 1, characterized in that The specific method for obtaining sleep EEG signals, constructing a dataset and dividing it into a training set, a test set, and a validation set is: Collect sleep EEG signals and divide them into a series of data segments with a duration of 30 seconds; Divide the data into a training set, a validation set, and a test set in a ratio of 8:1:1; Perform coarse-grained annotation on all data segments, classify each segment according to the sleep staging standard, which are the wake stage, N1 stage, N2 stage, N3 stage, and REM stage, and perform fine-grained annotation on the data segments in the test set, specifically annotating the occurrence intervals of sleep micro-events, and construct a sleep EEG signal database containing coarse-grained sleep staging and fine-grained micro-event annotation.

3. The weakly supervised dual-branch micro-event detection method according to claim 1, characterized in that The steps for obtaining sleep EEG signal data and preprocessing it include: Denoise the EEG signals to remove artifacts such as electrooculogram and electromyogram.

4. The weakly supervised dual-branch micro-event detection method according to claim 1, characterized in that The feature extraction network includes: A candidate region generation module, which uses a sliding window method to generate candidate signal segments with a certain overlap from the input signal as candidate regions; A feature extraction module, based on the DarkNet53 network structure of YOLOv3, extracts deep features through multiple convolutional layers and residual modules and inputs them into the first branch network and the second branch network.

5. The weakly supervised dual-branch micro-event detection method according to claim 4, characterized in that The first branch network converts the Euclidean distance between the deep features and the waveform prototype vectors into a similarity score, generates a similarity score matrix and inputs it into the clustering module, and after summing the similarity scores of all candidate segments, uses a fully connected network to obtain a sleep staging prediction.

6. The weakly supervised dual-branch micro-event detection method according to claim 5, characterized in that The clustering module calculates clustering clusters through the similarity score matrix.

7. The weakly supervised dual-branch micro-event detection method according to claim 6, characterized in that The specific method for the clustering module to calculate clustering clusters through the similarity score matrix is: The clustering module clusters the similarity score matrix in the candidate region dimension to find the clustering centers; Obtain the clustering clusters of candidate regions of micro-events existing in the input signal according to the clustering centers.

8. The weakly supervised dual-branch micro-event detection method according to claim 7, characterized in that The second branch network is transformed into a similarity score by calculating the Euclidean distance between the deep features and the waveform prototype vector, generating a similarity score matrix, and the second branch network and the first branch network share the same waveform prototype vector; The second branch network calculates the detection pseudo-labels and label weights of micro-events according to the clustering clusters, and trains the network by calculating the loss function with the similarity score matrix and the pseudo-labels.

9. The weakly supervised double-branch micro-event detection method according to claim 1, wherein The training of the network model adopts the Adam optimization method to dynamically adjust the learning rate.

10. The weakly supervised double-branch micro-event detection method according to claim 1, wherein The training of the network model adopts a composite loss function L, which is defined as follows: ； wherein are real number parameters respectively used to adjust the ratios between various functions, and are defined as follows: ； ； ； ； ； ； Among them, is the number of types of sleep stage classification, is the label of the electroencephalogram (EEG) signal sleep stage classification, represents the probability that the model predicts the sleep stage as , is the prototype feature obtained by network learning, is the deep feature vector extracted in the feature extraction network, is the number of input EEG signals, is the number of prototype features in the network. W is the weight parameter of the fully connected layer in the first branch network, is the pseudo-label weight.

Images