Small sample eeg signal recognition method, system and device based on multi-feature fusion

Abstract

The application provides a small sample electroencephalogram recognition method, system and device based on multi-feature fusion, which comprises the following steps: slicing a to-be-tested electroencephalogram according to a time axis to obtain a to-be-tested data set; performing micro-state feature extraction and common spatial pattern (CSP) feature extraction on each to-be-tested slice in the to-be-tested data set; splicing the micro-state feature and the CSP feature of each to-be-tested slice to form a fusion feature of the current to-be-tested slice; using a pre-trained classifier model to classify the fusion feature of each to-be-tested slice to identify whether each to-be-tested slice belongs to electroencephalogram of a target disease patient; and when the number of to-be-tested slices in the to-be-tested data set that are determined to belong to the electroencephalogram of the target disease patient is greater than a preset proportion of the total number of slices in the to-be-tested data set, the to-be-tested electroencephalogram is recognized as the electroencephalogram of the target disease patient. The application can accurately recognize the electroencephalogram of a depression patient and provide a diagnosis aid for doctors.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electroencephalogram signal processing, and in particular to a small sample electroencephalogram signal recognition method, system and device based on multi-feature fusion. BACKGROUND

[0002] Depression is one of the most common mental disorders, characterized by low mood, loss of interest or reduced energy. According to the report of the World Health Organization, more than 264 million people of all ages have depression. Depression has a serious impact on people's research, work and social life, and in the worst case, depression can lead to affected people committing suicide when the persistence or severity is persistent. The above information shows that depression has become a serious public health problem. In order to effectively prevent and treat depression, accurate diagnosis and identification are essential steps, and are also one of the problems to be solved in the clinic at present.

[0003] Electroencephalogram signal is a signal obtained by recording brain activity using electrophysiological indicators. Studies have shown that electroencephalogram signal can reflect the changes in brain function of patients with depression compared to healthy people. Therefore, how to accurately identify depression through electroencephalogram signal can more objectively provide doctors with diagnostic assistance, which is of great significance to improve the recognition probability of depression and strengthen the prevention and treatment of depression. SUMMARY

[0004] In view of the above problems, the present application is proposed in order to provide a small sample electroencephalogram signal recognition method, system and device based on multi-feature fusion, which overcomes the above problems or at least partially solves the above problems.

[0005] In one aspect of the present application, a small sample electroencephalogram signal recognition method based on multi-feature fusion is provided, the method comprising:

[0006] The test electroencephalogram signal is sliced according to the time axis to obtain a test data set;

[0007] The microstate feature extraction and common spatial pattern (CSP) feature extraction are respectively performed on each test slice in the test data set to obtain the microstate feature and the CSP feature of the test slice;

[0008] The microstate feature and the CSP feature of each test slice are spliced to form the fusion feature of the current test slice;

[0009] The pre-trained classifier model is used to classify the fusion feature of each test slice to identify whether each test slice belongs to the electroencephalogram signal of the patient with the target disease;

[0010] When the number of the to-be-tested slices in the to-be-tested data set determined as the electroencephalogram of the patient with the target disease is greater than the preset proportion of the total number of slices in the to-be-tested data set, the to-be-tested electroencephalogram is identified as the electroencephalogram of the patient with the target disease.

[0011] Optionally, the slicing of the to-be-tested electroencephalogram along the time axis to obtain the to-be-tested data set comprises:

[0012] The to-be-tested electroencephalogram is divided into a plurality of electroencephalogram slices in the time axis direction with a specified time length as a unit, and data augmentation is performed on each electroencephalogram slice by using the generative adversarial network technology to generate at least one generated data segment with statistical characteristics consistent with the original electroencephalogram of the corresponding slice, and the combination of each electroencephalogram slice and at least one generated data segment corresponding to each electroencephalogram slice is taken as the to-be-tested data set.

[0013] Optionally, before the to-be-tested electroencephalogram is divided into a plurality of electroencephalogram slices in the time axis direction with a specified time length as a unit, the method further comprises:

[0014] A band-pass filter of 0.1 Hz-70 Hz and a notch filter of 49 Hz-51 Hz are used to filter the to-be-tested electroencephalogram to eliminate line noise;

[0015] Time periods with a drift greater than a preset first threshold value in the to-be-tested electroencephalogram are removed, and an interpolation method is used to interpolate the to-be-tested electroencephalogram to improve the quality of the electroencephalogram data;

[0016] An ICA independent component analysis algorithm is used to separate independent components of ocular artifacts and / or muscle activity artifacts contained in the to-be-tested electroencephalogram;

[0017] A wavelet transform algorithm is applied to each separated independent component to identify and remove non-physiological artifact signals with a duration less than a preset second threshold value.

[0018] Optionally, the microstate feature extraction on each to-be-tested slice in the to-be-tested data set comprises:

[0019] Each to-be-tested slice in the to-be-tested data set is converted into a numpy array, and global field power (GFP) calculation is performed on the numpy array to obtain a corresponding GFP curve;

[0020] The GFP curve is processed by a clustering algorithm to cluster the topographic maps at the peak points of the GFP curve, to obtain electroencephalogram microstates at the slice level and microstate topographic maps, the electroencephalogram microstates of all to-be-tested slices in the to-be-tested data set are summarized to obtain microstate topographic maps at the group level, and the microstate topographic maps at the slice level and the microstate topographic maps at the group level are matched to obtain microstate segmentation vectors of each to-be-tested slice.

[0021] According to the microstate segmentation vector of each to-be-tested slice, the average duration, occurrence frequency, coverage range, global explained variance and transition probability of each microstate are calculated, principal component analysis is performed thereon to obtain a microstate feature vector, and the number of principal components selected by the principal component analysis is 3.

[0022] Optionally, the CSP feature extraction on each to-be-tested slice in the to-be-tested data set comprises:

[0023] Each electroencephalogram sample in a preset electroencephalogram data set is sliced according to a time axis to expand the samples, thereby obtaining a training data set, and the electroencephalogram data set comprises electroencephalogram samples of patients with a target disease and electroencephalogram samples of healthy people;

[0024] Sample data X and labels Y of sample slices of the training data set are extracted and converted into numpy arrays, wherein X is a three-dimensional array with a shape of [N, C, T], N is the number of training samples, T is the number of samples of a single sample, and C is the number of lead electrode channels; Y is a one-dimensional array with a length of N, containing label information, and the value is 0 or 1, representing a healthy person sample and a target disease patient sample respectively;

[0025] A csp model is constructed, sample data X and labels Y of each sample slice are input into the model for training, csp model matrix parameters are calculated, and a trained csp model is obtained;

[0026] Each to-be-tested slice data X' in the to-be-tested data set is extracted and converted into a numpy array, and each to-be-tested slice data X' is input into the trained csp model to extract csp features, and the dimension of the csp features is the same as that of the microstate features.

[0027] Optionally, the microstate features and the csp features of each to-be-tested slice are spliced to form fusion features of the current to-be-tested slice, which comprises:

[0028] The feature fusion of the microstate features and the csp features is realized by using a preset autoencoder, and the feature weight is automatically optimized to form a comprehensive feature representation.

[0029] Optionally, the method further comprises a step of pre-training a classifier model.

[0030] The pre-training of the classifier model comprises:

[0031] Each electroencephalogram sample in a preset electroencephalogram data set is sliced according to a time axis to expand the samples, thereby obtaining a training data set, and the electroencephalogram data set comprises electroencephalogram samples of patients with a target disease and electroencephalogram samples of healthy people;

[0032] The microstate feature extraction and the common spatial pattern (CSP) feature extraction are respectively performed on each sample slice in the training data set to obtain the microstate feature and the CSP feature of the sample slice.

[0033] The microstate feature and the CSP feature of each sample slice are spliced to form the fusion feature of the current sample slice.

[0034] The fusion feature of each sample slice in the training data set is used as the input of the classifier model, and the classification result of whether the input sample slice belongs to the electroencephalogram of the patient with the target disease or to the electroencephalogram of the healthy person is used for the classifier model learning and training to obtain the trained classifier model.

[0035] Optionally, each electroencephalogram sample in the electroencephalogram data set is recorded by using the 10-20 electrode lead positioning standard of the International Society for Electroencephalography, and a total of twenty-two useful leads are provided, which are Fp1, Fp2, Fpz, F3, F4, F7, F8, Fz, FCz, C3, C4, Cz, P3, P4, P7, P8, Pz, O1, O2 and Oz, wherein the Cz is set as a reference electrode.

[0036] In a second aspect, the present application further provides a small sample electroencephalogram recognition system based on multi-feature fusion, which comprises:

[0037] The slice module is configured to slice the to-be-tested electroencephalogram according to the time axis to obtain a to-be-tested data set.

[0038] The feature extraction module is configured to perform microstate feature extraction and common spatial pattern (CSP) feature extraction on each to-be-tested slice in the to-be-tested data set to obtain the microstate feature and the CSP feature of the to-be-tested slice.

[0039] The feature splicing module is configured to splice the microstate feature and the CSP feature of each to-be-tested slice to form the fusion feature of the current to-be-tested slice.

[0040] The classification module is configured to use the pre-trained classifier model to classify the fusion feature of each to-be-tested slice to identify whether each to-be-tested slice belongs to the electroencephalogram of the patient with the target disease.

[0041] The statistical analysis module is configured to identify the to-be-tested electroencephalogram as the electroencephalogram of the patient with the target disease when the number of to-be-tested slices determined to belong to the electroencephalogram of the patient with the target disease in the to-be-tested data set is greater than the preset proportion of the total number of slices in the to-be-tested data set.

[0042] In a third aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the small sample electroencephalogram recognition method based on multi-feature fusion.

[0043] In a fourth aspect, the present application also provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned small sample EEG signal recognition method based on multi-feature fusion when executing the computer program.

[0044] The small sample EEG signal recognition method, system and device based on multi-feature fusion provided by the embodiments of the present application increase the data sample size by using the slice sample amplification method, extract features by using the micro-state analysis and common spatial pattern (CSP) method, and extract discriminative features beneficial to the recognition of depression by using the feature fusion method, thereby realizing the high-precision EEG information recognition function of depression and providing technical support for clinical diagnosis, which has important significance for improving the recognition probability of depression and strengthening the prevention and treatment of depression.

[0045] The above description is only a summary of the technical solutions of the present application, in order to more clearly understand the technical means of the present application, the specific embodiments of the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Moreover, the same reference numerals in the attached drawings indicate the same or similar components. In the drawings:

[0047] Figure 1 The flowchart of the small sample EEG signal recognition method based on multi-feature fusion provided by the embodiments of the present application is shown in Figure 1;

[0048] Figure 2 The EEG signal channel lead schematic diagram is shown in Figure 2;

[0049] Figure 3 The EEG signal waveform after the preprocessing means of high-pass filtering, notch filtering and independent component analysis algorithm processing in the embodiments of the present application is shown in Figure 3;

[0050] Figure 4 The group level micro-state cluster topographic map in one embodiment of the micro-state analysis in the embodiments of the present application is shown in Figure 4;

[0051] Figure 5 The confusion matrix diagram of one embodiment of the micro-state analysis and CSP method classification in the embodiments of the present application is shown in Figure 5;

[0052] Figure 6This is a flowchart illustrating the specific process of training the classifier model proposed in this embodiment of the invention.

[0053] Figure 7 A structural block diagram of a small-sample EEG signal recognition system based on multi-feature fusion is provided for embodiments of the present invention. Detailed Implementation

[0054] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0055] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined.

[0057] Figure 1 The flowchart illustrating a method for identifying small-sample EEG signals based on multi-feature fusion according to an embodiment of the present invention is shown schematically. (Refer to...) Figure 1 The small-sample EEG signal recognition method based on multi-feature fusion in this invention specifically includes the following steps:

[0058] S11. Slice the EEG signal to be tested along the time axis to obtain the dataset to be tested.

[0059] In this embodiment, the operation of slicing the to-be-tested electroencephalogram signal along the time axis is implemented in the following manner: the to-be-tested electroencephalogram signal is divided into a plurality of electroencephalogram signal slices along the time axis direction in units of a specified time length, each slice optionally containing 2-3 seconds of electroencephalogram signal, data augmentation is performed on each electroencephalogram signal slice using a generative adversarial network technology to generate at least one generated data segment whose statistical features are consistent with the original electroencephalogram signal of the corresponding slice, and the combination of each electroencephalogram signal slice and at least one generated data segment corresponding to each electroencephalogram signal slice is taken as a to-be-tested data set.

[0060] Further, before dividing the to-be-tested electroencephalogram signal into a plurality of electroencephalogram signal slices along the time axis direction in units of a specified time length, the method further includes an operation of preprocessing the to-be-tested electroencephalogram signal to reduce interference from noise and other physiological activities. Specifically, a 0.1 Hz-70 Hz band-pass filter and a 49 Hz-51 Hz notch filter are used to filter the to-be-tested electroencephalogram signal to eliminate line noise; time periods with a drift greater than a preset first threshold in the to-be-tested electroencephalogram signal are removed, and an interpolation method is used to interpolate the to-be-tested electroencephalogram signal to improve the quality of the electroencephalogram data; an ICA independent component analysis algorithm is used to separate independent components of ocular artifacts and / or muscle activity artifacts contained in the to-be-tested electroencephalogram signal; and a wavelet transform algorithm is applied to each separated independent component to identify and remove non-physiological artifact signals with a duration less than a preset second threshold.

[0061] S12, microstate feature extraction and common spatial pattern (CSP) feature extraction are respectively performed on each to-be-tested slice in the to-be-tested data set to obtain microstate features and CSP features of the to-be-tested slice.

[0062] In this embodiment, the implementation of microstate feature extraction on each to-be-tested slice in the to-be-tested data set is as follows: each to-be-tested slice in the to-be-tested data set is converted into a numpy array, global field power (GFP) calculation is performed on the numpy array to obtain a corresponding GFP curve; clustering algorithm is used to cluster the GFP curve to cluster the topographic map at the peak point of the GFP curve, to obtain electroencephalogram microstates and microstate topographic maps at the slice level, and to aggregate the electroencephalogram microstates of all to-be-tested slices in the to-be-tested data set to obtain microstate topographic maps at the group level; the slice-level microstate topographic maps and the group-level microstate topographic maps are matched to obtain microstate segmentation vectors of each to-be-tested slice; and the average duration, occurrence frequency, coverage range, global explained variance, and transition probability of each microstate are calculated according to the microstate segmentation vector of each to-be-tested slice, principal component analysis is performed thereon to obtain a microstate feature vector, and the number of principal components selected by the principal component analysis is 3.

[0063] Further, the key to matching the slice-level microstate topography and the group-level microstate topography is to find the most similar microstate pattern. The present application achieves the matching of the slice-level microstate topography and the group-level microstate topography by calculating the similarity between each slice microstate topography and the group-level microstate topography, and selecting the most similar group-level microstate as the microstate segmentation vector of the slice. The calculation of the similarity can adopt various methods, such as the Euclidean distance, the cosine similarity or a more complex pattern recognition algorithm.

[0064] In the embodiment, the implementation manner of performing CSP feature extraction on each to-be-tested slice in the to-be-tested data set is as follows: each electroencephalogram signal sample in a preset electroencephalogram data set is sliced according to a time axis to expand the sample, to obtain a training data set, the electroencephalogram data set including electroencephalogram signal samples of patients with a target disease and electroencephalogram signal samples of healthy people; sample data X and a label Y of a sample slice of the training data set are extracted and converted into a numpy array, where X is a three-dimensional array with a shape of [N, C, T], N is the number of training samples, T is the number of samples of a single sample, and C is the number of lead electrode channels; Y is a one-dimensional array with a length of N, containing label information, and the value is 0 or 1, representing a healthy person sample and a target disease patient sample respectively; a csp model is constructed, the sample data X and the label Y of each sample slice are input into the model for training, the csp model matrix parameters are calculated, and a trained csp model is obtained; each to-be-tested slice data X' in the to-be-tested data set is extracted and converted into a numpy array, and each to-be-tested slice data X' is input into the trained csp model to extract csp features, the CSP feature dimension being the same as the microstate feature dimension.

[0065] S13, the microstate features and the CSP features of each to-be-tested slice are spliced to form the fusion features of the current to-be-tested slice.

[0066] In the embodiment, the feature fusion of the microstate features and the CSP features can be implemented by using a preset autoencoder to automatically optimize the feature weights and form a comprehensive feature representation. The microstate features and the CSP features can also be directly spliced to implement multi-feature fusion of the microstate features and the CSP features. The autoencoder (Autoencoder) is a kind of neural network used to learn an effective representation (encoding) of data. Various implementations can be adopted, such as a standard fully connected autoencoder, a convolutional autoencoder, etc. The goal of feature weight optimization is to improve the classification performance of the model, that is, to make the model better distinguish different categories of data. The weight optimization can be achieved through a training process, in which the weights are adjusted by minimizing the reconstruction error (i.e. the difference between the input and the output) when training the autoencoder, and the specific weight optimization process can be automatically completed through backpropagation and gradient descent algorithm.

[0067] S14, using a pre-trained classifier model to classify the fusion features of each to-be-detected slice to identify whether each to-be-detected slice belongs to the brain electrical signal of a patient with a target disease, and the target disease can be depression or other mental problems. The input data of the classifier model is the fusion features of the brain electrical signal slice, and the output data is the probability that the current input brain electrical signal slice belongs to the brain electrical signal of a patient with a target disease or the probability that it belongs to a healthy person. Specifically, the classifier model can be trained using a support vector machine or a random forest model.

[0068] S15, when the number of to-be-detected slices in the to-be-detected data set that are determined to belong to the brain electrical signal of a patient with a target disease is greater than a preset proportion of the total number of slices in the to-be-detected data set, the to-be-detected brain electrical signal is identified as the brain electrical signal of a patient with a target disease.

[0069] Specifically, after the splicing features of the to-be-detected slice are sent into the trained classifier, a classification probability P is obtained, and if P is greater than a preset value p, such as 60%-80%, the slice is classified as positive and belongs to a patient with depression. If the number of positive slices is greater than a preset proportion, such as 80%-90%, of the total number of slices in the sample, the brain electrical signal sample is considered to belong to a patient with depression.

[0070] The small sample brain electrical signal recognition method based on multi-feature fusion provided by the embodiment of the present application increases the data sample size through the slice sample expansion method, extracts features using the micro-state analysis and common spatial pattern csp method, and extracts discriminative features beneficial to recognizing depression in a feature fusion manner, thereby realizing high-precision depression brain electrical information recognition function, providing technical support for clinical diagnosis, and having important significance for improving the recognition probability of depression and strengthening the prevention and treatment of depression.

[0071] In order to overcome the problems of too many required training samples and low recognition accuracy in the existing brain electrical signal recognition depression technology, the embodiment of the present application provides a small sample brain electrical signal recognition method based on multi-feature fusion. The method selects appropriate feature extraction parameters to extract features from the preprocessed brain electrical signal through two feature extraction algorithms of micro-state analysis and common spatial pattern (CSP) to find a feature space that effectively distinguishes patients with depression from normal people, thereby ensuring the recognition accuracy. The problem of insufficient samples is solved by expanding the learning samples through the sample slicing method.

[0072] In the embodiment, the pre-training of the classifier model specifically includes the following implementation steps:

[0073] S21, each brain electrical signal sample in a preset brain electrical data set is sliced according to a time axis to expand the sample, thereby obtaining a training data set, and the brain electrical data set includes brain electrical signal samples of patients with a target disease and brain electrical signal samples of healthy people.

[0074] S22, respectively, microstate feature extraction and common spatial pattern CSP feature extraction are performed on each sample slice in the training data set to obtain the microstate feature and CSP feature of the sample slice.

[0075] S23, the microstate feature and CSP feature of each sample slice are spliced to form the fusion feature of the current sample slice;

[0076] S24, the fusion feature of each sample slice in the training data set is used as the input of the classifier model, and the current input sample slice belongs to the electroencephalogram signal of the target disease patient or the electroencephalogram signal of the healthy person is taken as the classification result to perform classifier model learning training, and a trained classifier model is obtained.

[0077] In application, when the spliced feature of the to-be-tested sample slice in the test sample set is input into the trained classifier, a classification probability P is obtained, if P is greater than a preset value p, the slice is classified as positive, belonging to a depression patient. If the number of positive slices is greater than a preset proportion of the total number of sample slices in the test sample set, it is considered that the electroencephalogram signal sample belongs to a depression patient.

[0078] In the embodiment, a plurality of electroencephalogram signal samples are obtained to form an electroencephalogram data set, the samples include healthy person samples and depression patient samples, and each sample includes 3-5 minutes of resting state electroencephalogram signal data.

[0079] The plurality of electroencephalogram signal samples are recorded by using the 10-20 electrode lead positioning standard calibrated by the International Electroencephalogram Society, and there are twenty-two useful leads, such as Figure 2Fp1, Fp2, Fpz, F3, F4, F7, F8, Fz, FCz, C3, C4, Cz, P3, P4, P7, P8, Pz, O1, O2, Oz, wherein Cz is set as the reference electrode. The sampling frequency of the acquired EEG signal sample is set to 1000hz, the parameter of the online high-pass filter is set to 0.5Hz, and the final EEG signal file is saved in the.mat format. In the application, the EEG signal is derived from the EEG signals of 87 volunteers in a resting state, including 50 patients and 37 healthy control persons, wherein the patients with depression have all been consulted by at least three psychiatrists, and have not been treated with psychotropic drugs in the past two weeks. All subjects remained conscious and in good mental state throughout the collection process. Before arriving at the laboratory, the subjects were first washed and dried, adjusted to a good self-state, then relaxed in a chair and adjusted to a comfortable sitting position. During the collection period, the researchers explained the entire test procedure and precautions to the subjects and answered their questions until the subjects fully understood the experimental process. Then, the subjects were fitted with an EEG cap, and physiological saline with potassium chloride was dropped into the cathode sponge to reduce impedance. After the average resistance of the subject's scalp was <10 kΩ and the EEG signal was stable, the collection of resting-state EEG data for 5 minutes with eyes closed was started. During the test, the subjects were instructed to avoid blinking as much as possible, and to keep their heads and bodies as still as possible. The subjects remained conscious throughout the test until the EEG data collection was completed.

[0080] Then, the EEG signal is preprocessed and the data set is made, specifically including the following steps:

[0081] 1. Traverse the.mat data file in the EEG signal sample, and import the original EEG data.

[0082] 2. Through electrode positioning, remove the useless electrode signal, and set the whole brain average reference potential.

[0083] 3. Filter the data sample, set a band-pass filter of 0.1Hz-70Hz and a notch filter of 49Hz-51Hz, and eliminate the line noise.

[0084] 4. Browse the EEG signal data, eliminate the time period with too much drift, and improve the data quality by using the interpolation method.

[0085] 5. Remove artifacts by using the joint method of ICA independent component analysis and waveform decorrelation: use ICA to separate the independent components that may contain artifacts such as eye movement and muscle activity, and then apply wavelet transform to each independent component to identify and remove short-term, non-physiological burst artifacts such as electrode jumps or muscle twitches.

[0086] 6. The resting state electroencephalogram data is segmented into 2 seconds, data augmentation is performed using generative adversarial network (GAN) technology to generate new data segments with consistent statistical characteristics as the original electroencephalogram data, bad segments are deleted, electroencephalogram data with a length of more than 180 seconds is saved, and saved as a.set format file to ensure compatibility with the EEGLAB toolbox for subsequent in-depth analysis.

[0087] In the embodiment, the waveform of the preprocessed electroencephalogram signal can not only be linearly corrected, but also can eliminate eye movement artifacts and part of bad lead components. Figure 3 The waveform of the electroencephalogram signal processed by the pre-processing means such as high-pass filtering, notch filtering and independent component analysis algorithm.

[0088] In the embodiment of the application, the microstate feature extraction of each sample slice in the training data set in step S22 includes the following steps:

[0089] Each sample slice in the given training data set is converted into a numpy array, and the corresponding GFP curve is obtained by performing GFP calculation.

[0090] In the embodiment, the GFP calculation is performed according to the following formula, wherein K represents the number of electroencephalogram electrodes, K is 22 in the embodiment, V i (t) represents the voltage of the i-th electrode at the t time point, V mean (t) represents the average value of the instantaneous voltage between the electrodes.

[0091]

[0092] The GFP curve is processed by a clustering algorithm, the topography of the GFP curve peak is clustered, the microstate vector and the microstate topography are obtained, each sampling point is classified into a corresponding microstate, and the microstate segmentation vector of the electroencephalogram signal is obtained. In addition, factor analysis or PCA analysis can also be performed on the EEG data at the GFP peak time to extract components representing the main variability, which can be regarded as candidates for microstates.

[0093] In the embodiment, first, the clustering of the microstate topography of a single sample slice is performed to obtain the electroencephalogram microstate at the slice level, then the electroencephalogram microstates of all slices of the data set are summarized to obtain the electroencephalogram microstate at the group level, and finally the microstate topography at the slice level and the microstate topography at the group level are matched to obtain the microstate segmentation sequence of the slice. Figure 4 The microstate topography of the electroencephalogram is shown in the figure. In the embodiment, the clustering method is k-means clustering, and the number of clusters is 4.

[0094] According to the obtained microstate segmentation vector of the brain electrical signal, the average duration, occurrence frequency, coverage range, global explained variance and transition probability of each microstate are calculated, principal component analysis is performed on the microstate feature vector. The average duration is the average time length of each microstate when it appears; the occurrence frequency is the average number of times that the microstate dominates every 0.1 seconds; the coverage range is the proportion of time covered by each microstate in the entire analysis period; the global explained variance is the percentage of the total variance explained by a given microstate; and the transition probability is the proportion of the number of times that one microstate switches to another microstate to the total number of times of switching between all microstates.

[0095] In this embodiment, the number of principal components selected by the principal component analysis is 3.

[0096] In the embodiment of the application, the step S22 of performing common spatial pattern (CSP) feature extraction on each sample slice in the training data set respectively includes the following steps:

[0097] 1. Extract the preprocessed brain electrical signal data X and label Y, and convert them into numpy arrays, wherein X is a three-dimensional array with a shape of [N, C, T], N is the number of training samples, T is the number of samples of a single sample, and C is the number of lead electrode channels; Y is a one-dimensional array with a length of N, containing label information, and the value is 0 or 1, representing a healthy person sample and a target disease patient sample respectively.

[0098] 2. Construct a csp model, input the training data and label into the model, calculate the csp model matrix related parameters, and obtain the trained csp model.

[0099] In this embodiment, the method for constructing the CSP model and calculating the model related parameters is as follows:

[0100] (1) Let the patient sample of the depressed person and the healthy person sample be X1 and X2 respectively, which are space-time signals of CxT, C represents the number of channels, and T represents the number of sampling points. Normalize the two.

[0101] (2) Calculate the covariance matrix of the two Obtain the mixed covariance matrix In the formula, is the average covariance matrix of all depressed samples and healthy samples. Further, when calculating the covariance matrix of the two types of samples, a regularization term can be introduced to improve the condition number. Alternatively, a shrinkage method can be used.

[0102] (3) Perform eigenvalue decomposition on the matrix R R = UλU T , and obtain the whitening matrix Wherein, λ is a diagonal matrix composed of eigenvalues of matrix R. Further, when performing eigenvalue decomposition on the covariance matrix, an iterative-based optimization algorithm can be used to improve the accuracy and stability of the eigenvalue decomposition.

[0103] (4) Transform R1, R2 to obtain S1 = PR1P T , S2 = PR2P T Perform principal component decomposition on both to obtain B1 = B2 = V, and λ1 + λ2 = I. Wherein, λ1 is a diagonal matrix of eigenvalues of the transformed covariance matrix S1 of the depression sample, and λ2 is a diagonal matrix of eigenvalues of the transformed covariance matrix S2 of the healthy sample.

[0104] (5) Arrange the eigenvalues in λ1 in descending order and the eigenvalues in λ2 in ascending order, and it can be deduced that λ1 = diag (I1σ M 0), λ2 = diag (0σ M I2), I1 is the front part of λ1 or λ2 composed of some eigenvalues close to 1 (I1), the middle part is composed of some larger eigenvalues (σ M ), and the last part is composed of some eigenvalues close to 0. Finally, the projection matrix W = B T P.

[0105] 3. The sample to be converted is converted into a numpy array according to the corresponding format and sent into the trained csp model to extract csp features.

[0106] In this embodiment, the sample to be converted is X, and the feature extraction process is Wherein, f is the feature vector.

[0107] In this embodiment, the CSP feature dimension is set to 3.

[0108] After splicing the microstate features and the CSP features, a feature vector with a dimension of 6 is obtained. Feature extraction is performed on the training set samples and the test set samples to obtain training set features and test set features. The training set features and the labels are sent into two classifiers, support vector machine SVM and random forest RF, for training. The trained classifiers are used to classify the test set features, and the classification result of each test person's electroencephalogram signal is obtained by voting the classification results of the electroencephalogram signal slices. The classification accuracy of the two classifiers is 100%. Figure 5

[0109] In the above test scenario, the specific implementation process is as follows. Figure 6 ​The electroencephalogram signal data is divided into a training set and a test set, wherein the training set includes electroencephalogram signal slices of 15 patients and 15 healthy people, and the test set includes electroencephalogram signal slices of 35 patients and 22 healthy people. Under the condition that only 15 patients with depression and 15 healthy people are used as training samples, the remaining 35 patients with depression and 22 healthy people are completely correctly classified, which further illustrates the effectiveness of the feature extraction method, sample expansion method and voting classification method provided by the method in the depression recognition task.

[0110] In this embodiment, the specific implementation modes of the microstate feature extraction and the common spatial pattern CSP feature extraction of each to-be-tested slice in the to-be-tested data set are consistent with the above-mentioned implementation modes of the training set sample slice feature extraction, and details are not described herein.

[0111] The small sample electroencephalogram signal depression recognition algorithm based on the microstate analysis and CSP method provided by the present application extracts discriminative features beneficial to recognizing depression through the microstate analysis and CSP features, and successfully realizes the small sample depression recognition task under the condition that the training sample is limited through the sample expansion method, thereby providing technical support for clinical diagnosis.

[0112] For the method embodiments, in order to simply describe, all are expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the action sequence described, because according to the embodiments of the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of the present application.

[0113] In addition, the embodiments of the present application also provide a small sample electroencephalogram signal recognition system based on multi-feature fusion. Referring to Figure 7 The small sample electroencephalogram signal recognition system based on multi-feature fusion provided by the embodiments of the present application includes functional modules for realizing the small sample electroencephalogram signal recognition method based on multi-feature fusion as described in any one of the above. Specifically, the system includes a slice module 701, a feature extraction module 702, a feature splicing module 703, a classification module 704 and a statistical analysis module 705, wherein:

[0114] The slice module 701 is used to slice the to-be-tested electroencephalogram signal according to the time axis to obtain a to-be-tested data set;

[0115] The feature extraction module 702 is used to respectively perform microstate feature extraction and common spatial pattern CSP feature extraction on each to-be-tested slice in the to-be-tested data set to obtain the microstate feature and the CSP feature of the to-be-tested slice;

[0116] The feature splicing module 703 is configured to splice the microstate feature and the CSP feature of each to-be-tested slice to form a fusion feature of the current to-be-tested slice.

[0117] The classification module 704 is configured to classify the fusion feature of each to-be-tested slice by using a pre-trained classifier model to identify whether each to-be-tested slice belongs to the electroencephalogram of the target disease patient.

[0118] The statistical analysis module 705 is configured to identify the to-be-tested electroencephalogram as the electroencephalogram of the target disease patient when the number of to-be-tested slices in the to-be-tested data set that are determined to belong to the electroencephalogram of the target disease patient is greater than a preset proportion of the total number of slices in the to-be-tested data set.

[0119] In the embodiment of the present application, the system further comprises a model training module not shown in the drawings, which is configured to pre-train the classifier model.

[0120] The model training module is specifically configured to slice each electroencephalogram sample in a preset electroencephalogram data set according to a time axis to expand the sample, to obtain a training data set, wherein the electroencephalogram data set includes electroencephalogram samples of target disease patients and electroencephalogram samples of healthy people; to perform microstate feature extraction and common spatial pattern (CSP) feature extraction on each sample slice in the training data set respectively, to obtain the microstate feature and the CSP feature of the sample slice; to splice the microstate feature and the CSP feature of each sample slice to form a fusion feature of the current sample slice; and to use the fusion feature of each sample slice in the training data set as an input of the classifier model, to use whether the current input sample slice belongs to the electroencephalogram of the target disease patient or belongs to the electroencephalogram of the healthy person as a classification result to learn and train the classifier model, to obtain the trained classifier model.

[0121] For the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts are described in the part of the method embodiment, and the corresponding technical effects are achieved.

[0122] In addition, the embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the small sample electroencephalogram recognition method based on multi-feature fusion.

[0123] If the small sample EEG signal recognition method based on multi-feature fusion in the embodiment is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer-readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium can include any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0124] In addition, the embodiment of the present application also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the small sample EEG signal recognition method based on multi-feature fusion as described above when executing the computer program. For example Figure 1 The processor implements the functions of each module / unit in the above-mentioned small sample EEG signal recognition system embodiment based on multi-feature fusion when executing the computer program, for example Figure 7 The slicing module 701, the feature extraction module 702, the feature splicing module 703, the classification module 704, and the statistical analysis module 705.

[0125] The small sample EEG signal recognition method, system, and device based on multi-feature fusion provided by the embodiment of the present application increase the data sample size by using the slicing sample expansion method, extract features by using the micro-state analysis and common spatial pattern (CSP) method, and extract discriminative features beneficial to recognizing depression by using the feature fusion method, thereby realizing high-precision EEG information recognition of depression and providing technical support for clinical diagnosis. It is of great significance to improve the recognition probability of depression and strengthen the prevention and treatment of depression.

[0126] Furthermore, those skilled in the art will recognize that, while certain embodiments described herein include certain features that are not included in other embodiments, combinations of features of the different embodiments are to be construed as being within the scope of the application and forming different embodiments. For example, any of the claimed embodiments can be used in any combination.

[0127] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced by equivalent features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A small sample electroencephalogram recognition method based on multi-feature fusion, characterized in that, The method comprises: slicing the to-be-tested electroencephalogram along a time axis to obtain a to-be-tested data set; extracting microstate features and common spatial pattern (CSP) features from each to-be-tested slice in the to-be-tested data set to obtain microstate features and CSP features of the to-be-tested slice; splicing the microstate features and the CSP features of each to-be-tested slice to form a fusion feature of the to-be-tested slice; classifying the fusion feature of each to-be-tested slice by using a pre-trained classifier model to identify whether each to-be-tested slice belongs to electroencephalogram of a patient with a target disease; when the number of to-be-tested slices in the to-be-tested data set that are determined to belong to the electroencephalogram of the patient with the target disease is greater than a preset proportion of the total number of slices in the to-be-tested data set, the to-be-tested electroencephalogram is identified as the electroencephalogram of the patient with the target disease.

2. The method of claim 1, wherein, The slicing of the to-be-tested electroencephalogram along the time axis to obtain the to-be-tested data set comprises: dividing the to-be-tested electroencephalogram into a plurality of electroencephalogram slices in a time axis direction at a specified time length as a unit, performing data enhancement on each electroencephalogram slice by using a generative adversarial network technology to generate at least one generated data segment with consistent statistical features and original electroencephalogram of the corresponding slice, and taking a combination of each electroencephalogram slice and at least one generated data segment corresponding to each electroencephalogram slice as the to-be-tested data set.

3. The method of claim 2, wherein, Before dividing the to-be-tested electroencephalogram into a plurality of electroencephalogram slices in the time axis direction at a specified time length as a unit, the method further comprises: filtering the to-be-tested electroencephalogram by using a 0.1 Hz-70 Hz band-pass filter and a 49 Hz-51 Hz notch filter to eliminate line noise; removing a time period in the to-be-tested electroencephalogram with a drift greater than a preset first threshold value, and performing interpolation on the to-be-tested electroencephalogram by using an interpolation method to improve the quality of the electroencephalogram data; separating independent components of ocular artifacts and / or muscle activity artifacts contained in the to-be-tested electroencephalogram by using an ICA independent component analysis algorithm; applying a wavelet transform algorithm to each separated independent component to identify and remove non-physiological artifact signals with a duration less than a preset second threshold value.

4. The method of claim 1, wherein, The microstate feature extraction of each to-be-tested slice in the to-be-tested data set comprises: transforming each to-be-tested slice in the to-be-tested data set into a numpy array, performing global field power (GFP) calculation on the numpy array to obtain a corresponding GFP curve; performing clustering processing on the GFP curve by using a clustering algorithm to cluster a topographic map at a peak point of the GFP curve, obtaining electroencephalogram microstates and a microstate topographic map at a slice level, and summarizing the electroencephalogram microstates of all to-be-tested slices in the to-be-tested data set to obtain a microstate topographic map at a group level; matching the microstate topographic map at the slice level and the microstate topographic map at the group level to obtain a microstate segmentation vector of each to-be-tested slice; calculating the average duration, occurrence frequency, coverage range, global explained variance and transition probability of each microstate according to the microstate segmentation vector of each to-be-tested slice, performing principal component analysis on the same, and obtaining a microstate feature vector, wherein the number of principal components selected by the principal component analysis is 3.

5. The method of claim 1, wherein, The CSP feature extraction is performed on each to-be-tested slice in the to-be-tested data set, including: Each electroencephalogram signal sample in the preset electroencephalogram data set is sliced along a time axis to expand the sample, to obtain a training data set, and the electroencephalogram data set includes electroencephalogram signal samples of patients with a target disease and electroencephalogram signal samples of healthy people; Sample data X and a label Y of a sample slice of the training data set are extracted and converted into a numpy array, wherein X is a three-dimensional array with a shape of [N, C, T], N is the number of training samples, T is the number of samples of a single sample, and C is the number of lead electrode channels; Y is a one-dimensional array with a length of N, containing label information, and the value is 0 or 1, representing a healthy person sample and a patient sample with a target disease, respectively; A csp model is constructed, sample data X and the label Y of each sample slice are input into the model for training, the csp model matrix parameters are calculated, and the trained csp model is obtained; Each to-be-tested slice data X' in the to-be-tested data set is extracted and converted into a numpy array, and each to-be-tested slice data X' is input into the trained csp model to extract csp features, and the csp feature dimension is the same as the microstate feature dimension.

6. The method of claim 1, wherein, The microstate feature and the CSP feature of each to-be-tested slice are spliced to form the fusion feature of the current to-be-tested slice, including: The feature fusion of the microstate feature and the CSP feature is realized by using a preset automatic encoder, and the feature weight is automatically optimized to form a comprehensive feature representation.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes the step of pre-training a classifier model; The pre-training of the classifier model includes: Each electroencephalogram signal sample in the preset electroencephalogram data set is sliced along a time axis to expand the sample, to obtain a training data set, and the electroencephalogram data set includes electroencephalogram signal samples of patients with a target disease and electroencephalogram signal samples of healthy people; The microstate feature and the CSP feature of each sample slice are spliced to form the fusion feature of the current sample slice; The fusion feature of each sample slice in the training data set is used as the input of the classifier model, and the brain electrical signal of the current input sample slice belongs to the target disease patient or the brain electrical signal of the healthy person is used as the classification result for the classifier model learning training, to obtain the trained classifier model. Each electroencephalogram signal sample in the electroencephalogram data set is recorded by using the 10-20 electrode lead positioning standard of the International Society for Electroencephalography, and there are twenty-two useful leads, which are Fp1, Fp2, Fpz, F3, F4, F7, F8, Fz, FCz, C3, C4, Cz, P3, P4, P7, P8, Pz, O1, O2, and Oz, wherein Cz is set as a reference electrode.

8. The method of claim 7, wherein, The system includes:

9. A small sample electroencephalogram signal recognition system based on multi-feature fusion, characterized by, A slicing module for slicing a to-be-tested electroencephalogram signal along a time axis to obtain a to-be-tested data set; ​ The feature extraction module is configured to perform microstate feature extraction and common spatial pattern (CSP) feature extraction on each of the to-be-tested slices in the to-be-tested data set, to obtain microstate features and CSP features of the to-be-tested slices. The feature splicing module is configured to splice the microstate features and the CSP features of each to-be-tested slice to form a fusion feature of the to-be-tested slice. The classification module is configured to classify the fusion feature of each to-be-tested slice by using a pre-trained classifier model, to identify whether each to-be-tested slice belongs to the electroencephalogram of the patient with the target disease. The statistical analysis module is configured to identify the to-be-tested electroencephalogram as the electroencephalogram of the patient with the target disease when the number of to-be-tested slices determined to belong to the electroencephalogram of the patient with the target disease in the to-be-tested data set is greater than a preset proportion of the total number of slices in the to-be-tested data set.

10. A computer device, comprising: A computer program product comprising a memory, a processor and a computer program stored on the memory and loadable on the processor, the processor implementing the steps of the method according to any one of claims 1 to 8 when executing the computer program.

Images