A human body state evaluation method based on a deep learning hybrid model

Abstract

A human state evaluation method based on a deep learning hybrid model, comprising: acquiring sensor space multi-channel electroencephalogram signals under anesthetic state and preprocessing the acquired electroencephalogram signals; electroencephalography source imaging and reconstruction, and partition calculation of activity distribution of each brain region for electroencephalography source reconstruction to generate source space electroencephalogram signals; calculating an unbiased weighted phase delay index connection matrix, using permutation test between two groups of people to find out frequency bands with significant differences, and using the frequency bands with significant differences for subsequent analysis; based on the frequency bands with significant differences, training a feature dissociation double-encoder convolutional autoencoder model; connecting the trained autoencoder model with a convolutional neural network model to form a deep learning hybrid model, training the model, and finally performing model evaluation; the present application analyzes sensor electroencephalogram signals, uses a deep learning hybrid model for identification and classification, reduces useless information in the data, and improves the accuracy of the model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of machine learning and pattern recognition, and particularly relates to a human state evaluation method based on a deep learning hybrid model, which is used for accurate brain signal monitoring of patients during anesthesia. BACKGROUND

[0002] Electroencephalogram (EEG) is a method of recording brain activity using electrophysiological indicators. When the brain is active, it is formed by the summation of a large number of neuron synchronous post-synaptic potentials, and it records the change of the electric wave when the brain is active. It is the overall reflection of the electrical physiological activity of brain nerve cells on the cerebral cortex or scalp surface.

[0003] It is a great challenge for modern neuroscience to understand how the human brain reversibly produces and loses consciousness through the complex interaction of neural activity on multiple spatial and temporal scales. Although there has been rapid progress in scientific understanding of how brain networks produce consciousness in recent years, it has proven difficult to apply this knowledge to accurately track the transition to unconsciousness during general anesthesia, because this gradual process has considerable variability between individuals. Srivas C published in Srivas C, Stuart O, Ram A, et al. Brain Connectivity Dissociates Responsiveness from Drug Exposure during Propofol-Induced Transitions of Consciousness. [J]. PLoS computational biology, 2016, 12 (1). Using high-density electroencephalography, the changes in these networks were studied in healthy adults who were sedated with propofol, and by measuring their behavioral responsiveness and the sedative content in their blood, it was found that an amazing mechanism: the strength of the brain network in healthy adults before sedation can predict why some participants lose consciousness, while others do not, although their drug levels in the blood are similar. By revealing the potential characteristics of this variability, this mechanism can provide accurate brain monitoring during anesthesia and minimize intraoperative loss of consciousness.

[0004] The existing patent application "A motion intention prediction method based on brain network dynamic connection characteristics" (patent application number: CN202010211448.3) proposes a motion intention prediction method based on brain network dynamic connection characteristics, mainly including: constructing a time-varying dynamic Bayesian network model, converting the EEG signal matrix in the time domain into a directed weighted connection matrix between electrodes in the time domain; The paired t-test is performed on each electrode pair according to the defined motion intention state and the resting state, and the electrode pairs with significant differences in the two states are screened out, and the electrode pairs and time are reconstructed into a feature matrix; The feature matrix is reconstructed into a feature vector, and the feature vector is input into the trained classifier to output the prediction result of the motion intention. However, this method has some shortcomings, first, the significant difference of the EEG signal frequency band is not extracted, the EEG signal frequency band is long, and the full frequency band input analysis may cause redundancy of effective information and flooding of invalid information, second, the training of the model uses the random forest method, the training time is long, the calculation cost is large, the iteration training time is long, and the hardware requirements of the computer are high. SUMMARY

[0005] In order to overcome the defects of the above-mentioned prior art, the purpose of the present application is to provide a human state evaluation method based on a deep learning hybrid model, which establishes a deep learning hybrid model to classify and realize the classification of two types of people who are easy to be anesthetized and not easy to be anesthetized, has high calculation efficiency, improves accuracy, and only uses data in a clear state during testing, and only observes the classification effect of samples in a clear state to realize prediction.

[0006] In order to achieve the above-mentioned purpose, the technical scheme of the present application is:

[0007] A human state evaluation method based on a deep learning hybrid model, comprising the following steps:

[0008] Step 1: Obtain the sensor space multi-channel anesthetic state electroencephalogram signal and pre-process the obtained electroencephalogram signal;

[0009] Step 2: Electroencephalography and reconstruction, i.e. calculating the sensor space electroencephalogram signal to obtain its mapping in the electroencephalography space; and calculating the activity distribution of each brain region according to the Desikan-Killiany Atlas brain partition, performing electroencephalography reconstruction, and generating source space electroencephalogram signal;

[0010] Step 3: In all time domains, calculate the unbiased weighted phase delay index connection matrix of all people and all channels for different samples; use permutation test between the two groups of people to find out the frequency band with significant difference and the corresponding unbiased weighted phase delay index connection matrix;

[0011] Step 4: Train a feature deconvolution dual-encoder convolutional autoencoder model based on the unbiased weighted phase delay exponential connection matrix of frequency bands with significant differences.

[0012] Step 5: Connect the trained autoencoder model with the convolutional neural network model to form a deep learning hybrid model, train the model, and finally evaluate the model.

[0013] In step one, during the acquisition of multi-channel EEG signals in the sensor space, the acquired multi-channel EEG signals include two segments: one from a conscious state and the other from a moderately anesthetized state. The acquired multi-channel EEG signals in the sensor space are represented as follows: Where n represents whether the subject is susceptible to anesthesia, and m is the number of the Noc-guided EEG signal collected from the m-th subject during the experiment. Represents a real matrix. There is a Noc row and a t column, where t is the number of sampling points, and the sampling time is assumed to be t. t If the sampling frequency is F, then t = t t ×F.

[0014] Step one involves preprocessing the acquired EEG signals, including bandpass filtering, period segmentation, baseline correction, and removal of eye movement or muscle artifacts. During bandpass filtering, the retained sensor spatial EEG signals are processed. All EEG signals from all channels were filtered using a bandpass filter of 0.5–45 Hz. In the epoch division, all EEG signals from all channels were divided into multiple equal-length epochs, each numbered as an epoch. e 'e' represents the period number, and the maximum value of 'e' depends on the number of periods divided, with a maximum value of 'e'. max Each resulting epoch is numbered accordingly. e The EEG signal relative to the entire EEG signal Baseline correction was performed using the average voltage; eye movement or muscle artifacts were removed by calculating the EEG signal. Normalized variance was used to identify channels and periods of anomalous noise, which were then manually rejected or retained through visual inspection. The preprocessed sensor spatial EEG data were then obtained.

[0015] The specific process of brain electrical activity imaging and reconstruction in step two is as follows:

[0016] (1) Solve the EEG forward problem, that is, to establish a head model that can explain how the activity of neurons in the cerebral cortex is projected to the scalp through the brain volume effect and the conduction of the skull. The Freesurfer average based on magnetic resonance imaging (MRI) is used as the anatomical brain template, and the Brainstorms default parameters of the boundary element method (BEM) in OpenMEEG are used to generate the head model of each participant. The mathematical model of the scalp EEG signal is established as follows:

[0017]

[0018] The sensor EEG signal obtained after step one is represented by A, which is a zero lead domain matrix that simulates the distribution of neuron cells in the cerebral cortex and the changes in their post-synaptic potentials. X represents the dipole moment of the cortical current dipole, which simulates the distribution of neuron cells in the cerebral cortex and the changes in their post-synaptic potentials (source activity). noise represents the measurement noise.

[0019] (2) The minimum norm estimation method is used to solve the inverse problem.

[0020]

[0021] L2 regularization is used to further constrain the solution to find the solution X that minimizes the residual and energy in the source space. The regularization parameter λ is obtained using the L curve method to balance the influence of the two terms. In the case of L2 norm, the analytical solution of equation (2) gives the optimal estimate of the source distribution X, as shown in equation (3):

[0022]

[0023] When W is only a unit matrix, this solution is called the minimum norm estimation (MNE), which is the mapping of the sensor space EEG signal to the EEG source space. A T represents the inverse matrix of the lead field.

[0024] In step two, the activity distribution of each brain region is calculated according to the Desikan-Killiany Atlas brain partition, and the EEG source reconstruction is performed to generate the source space EEG signal. Specifically, the number of dipoles in each brain region is classified according to the Desikan-Killiany Atlas brain partition, and the average value of the measured values of the dipoles in the same time in this region is taken as the output of the whole brain partition, so as to obtain the source distribution matrix of a single subject EEG signals mapped from sensor space to source space, N c EEG signals from sensors The number of channels in source space, i.e. the number of regions of interest after parcellation according to Desikan-Killiany Atlas; T represents the number of sampling points.

[0025] The unbiased weighted phase lag index connection matrix in step three is calculated as follows:

[0026] x and y are EEG signals from two different rows, representing different channels of EEG signals in source space. For different epochs in EEG signals x and y from two different channels e The next sample segment is represented as signal and and time-frequency analysis is performed thereon. The length of time bin is set, and the time bin number is bin ft The length of frequency bin is set, and the frequency bin number is bin fi The cross spectrum is obtained by multiplying the Fourier transform of signal and wherein represents the complex conjugate of The cross spectrum is calculated using the same epoch of signals x and y e The same bin ft value of the next segment corresponds completely in time scale. The phase lag index (PLI) of different channel signals x and y is defined as follows:

[0027]

[0028] represents the imaginary part of cross spectrum sign is a sign function, and E represents the mean value of all bins ft

[0029] represents the absolute value of

[0030]

[0031] Since PLI has a positive bias, the calculation formula of unbiased phase lag index is as follows:

[0032]

[0033] The indexes e j and e k represent different bins​​​ft The time period numbering, the weighted phase lag index (WPLI) is less sensitive to noise than the phase lag index, and is defined as:

[0034]

[0035] Similar to PLI, WPLI also exhibits a positive deviation. DWPLI is calculated as follows:

[0036]

[0037] In the formula, j and k represent bin ft The values ​​are accumulated in the time bins. When calculating DWPLI, it is required that each frequency bin be calculated within the same frequency bin. fi corresponding Different channels Combined, to generate matrix Elements in the matrix express The EEG signals of the i-th channel and the j-th channel in a certain epoch e bin fi This frequency was calculated within the chamber. n indicates whether the subject is susceptible to anesthesia, and m is the number of the m-th subject.

[0038] In step three, a permutation test between the two groups of people is used to identify frequency bands with significant differences. Specifically, this involves first solving the bin for each frequency band. fi of The mean of the matrix, binning the values ​​of the same type. fi Below Each element in Add and divide by N c ×(N c -1), to obtain any frequency bin of any subject segment sample. Each epoch e Get multiple Value; will use the same bin fi Value Form a sequence

[0039] For each sequence Perform a permutation test, with the number of permutations being times. Within each permutation, ... Randomly shuffle the sequence, but do not shuffle the labels. Calculate the mean value of the susceptible population for anesthesia based on the indices of the randomly shuffled sequence. forwardAt the same time, the value of the non-easy-to-anesthetize population in the sequence is taken according to the index of the randomly shuffled sequence to obtain the mean value mean backpro , the difference value minus timei , timei represents the minus obtained by the i-th permutation timei , all minus timei are sorted in ascending order to obtain the MINUS sequence, and the MINUS sequence has a total of TIME elements;

[0040] The confidence coefficient trust is set, the upper threshold value is solved and the lower threshold value The calculation formula is:

[0041]

[0042]

[0043] In the above formula, MINUS{floor[TIME×(trust / 2)]} represents taking the element of the MINUS sequence whose index is floor[TIME×(trust / 2)], and floor represents the floor function.

[0044] All belonging to the same category and the same frequency bin are taken to obtain , and the elements of the Dec sequence are , and are compared with and When is greater than or less than , it is considered that the frequency bin corresponding to the frequency bin numbered bin fi is a frequency bin with significant differences, and the frequency bin with significant differences is extracted, and the total number of the selected frequency bins is The sensor electroencephalogram signal in the awake state and the sensor electroencephalogram signal in the moderate anesthesia state can be extracted to have significant differences in the frequency band. The corresponding matrix of the sensor electroencephalogram signal in the awake state extracted to have significant differences in the frequency band is The corresponding matrix of the sensor electroencephalogram signal in the moderate anesthesia state extracted to have significant differences in the frequency band is The set parameter is set to the state of the collected sensor spatial electroencephalogram signal, and has two values of awake and moderate, which correspond to the awake state data and the moderate anesthesia state data respectively, so the matrix can be changed to

[0045] The fourth step is specifically as follows:

[0046] (1) Data conversion. According to the input requirements of the encoder, the non-diagonal elements in the matrix are taken half to deform into a one-dimensional vector, and the obtained one-dimensional vector is

[0047] (2) Establish a feature dissociation double-encoder convolutional autoencoder model. The model has two parallel encoders and one decoder. The two encoders are respectively used to extract individual features and pattern features in the data. The encoder for extracting individual features is named E p , and the encoder for extracting pattern features is named E s . Before the decoder, the individual features and the pattern features are added and fused, and the decoder reconstructs the fused features into the original data. The decoder is represented by D.

[0048] (3) Construct a loss function. The one-dimensional vector is input into two encoders E p and E s respectively, and two vectors are obtained respectively, which are and The two vectors have the same size. The extracted individual features and the pattern features are fused, and the fused data is input into the decoder D to obtain the reconstructed data The reconstructed data is used to compare with the original input data to calculate the reconstruction loss, and the calculation formula is:

[0049]

[0050]

[0051] At the same time, a cross-loss function is constructed. The one-dimensional vectors generated by the matrix do not belong to the same group of people, that is, the values of n are different The one-dimensional vector generated by the matrix is The calculation formula of the cross-loss function is:

[0052]

[0053] For another one-dimensional vector which is of the same category as but has a different set state, a triplet loss function is constructed, and the calculation formula is:

[0054]

[0055]

[0056] Alpha is a fixed value, and the total loss Loss function is calculated as follows:

[0057] Loss = Loss m,n + C Loss + TS Loss (15)

[0058] The total loss Loss is propagated in the reverse gradient, and the model parameters, including the parameters of E p , E s and D, are optimized.

[0059] The step five process specifically includes:

[0060] (1) Constructing a deep learning hybrid model:

[0061] Connect E s trained in step four with the convolutional neural network model, that is, use E s (X m,n ) as the input of the convolutional neural network model, and the convolutional neural network model is composed of a first convolutional layer, a first batch normalization layer, a first average pooling layer, a second convolutional layer, a second batch normalization layer, a second average pooling layer, a flat layer, a first full connection layer, a third batch normalization layer, a second full connection layer, a fourth batch normalization layer and a third full connection layer in sequence.

[0062] (2) Training the model:

[0063] Use the corresponding matrix of the frequency band with significant differences in the clear state and the corresponding one-dimensional vector obtained by transforming the matrix of the frequency band with significant differences extracted from the sensor electroencephalogram signal in the moderate anesthesia state as the data for model training, and the label only contains whether the one-dimensional vector belongs to the easy anesthesia population.

[0064] (3) Model evaluation:

[0065] The M-fold cross-validation method is used for evaluation, and only the corresponding one-dimensional vector of the corresponding matrix of the frequency band with significant differences in the clear state is used for testing, that is, only the classification effect of the sample in the clear state is observed, and the above operation is repeated M times, and finally the average recognition accuracy is calculated.

[0066] Compared with the prior art, the present application has the following advantages:

[0067] (1) In step two, the present application is based on the neurophysiological theory, and the brain electrical source space imaging is carried out to establish the source space electroencephalogram, which is beneficial to reduce the volume conduction effect in the electroencephalogram transmission process and the noise interference in the collection process.

[0068] (2) In step three, the influence of different frequencies on phase synchronization is considered in the DWPLI calculation, the statistical bias is corrected, the volume conduction is robust, the calculation speed is fast, the sensitivity to noise is low, and the analysis of electroencephalogram has advantages.

[0069] (3) In step three, the present application uses the permutation test to select the frequency band with significant difference, and the subsequent analysis is also based on the frequency band with significant difference, so that the effective input of the model is more, and the analysis result is more accurate.

[0070] (4) In step four, the present application separates the individual features and pattern features of the subjects in the feature separation double-encoder convolutional autoencoder model, and uses the output of the pattern feature as the input of the convolutional neural network model in step five, which reduces the useless information in the data and improves the accuracy of the model.

[0071] (5) In step five, the present application establishes a deep learning hybrid model to classify, which has high calculation efficiency and improves the accuracy, and only uses the data in the awake state during testing, and only observes the classification effect of the samples in the awake state to realize the prediction, which has great significance for predicting the human state before anesthesia.

[0072] In summary, compared with the prior art, the present application realizes the classification of the two types of people who are easy to be anesthetized and not easy to be anesthetized by establishing a deep learning hybrid model for classification, which has high calculation efficiency and improves the accuracy, and only uses the data in the awake state during testing, and only observes the classification effect of the samples in the awake state to realize the prediction. BRIEF DESCRIPTION OF DRAWINGS

[0073] Figure 1 The flowchart of the present application.

[0074] Figure 2 (a) is a scalp electroencephalogram collection device diagram, Figure 2 (b) is an electroencephalogram channel distribution diagram.

[0075] Figure 3 The experimental scheme diagram of the sensor space electroencephalogram signal collection.

[0076] Figure 4 The feature separation double-encoder autoencoder model diagram. DETAILED DESCRIPTION

[0077] The flowchart of the present application and the advantages will be described in detail below.

[0078] The specific experiment of the application is based on Matlab R2019b under the Windows 10 (64-bit) operating system for electroencephalogram preprocessing, data expansion and source imaging. The network training is based on the Tensor Flow2.6.0 framework, Keras as the front-end API, and Python3.9 as the virtual environment of the compiler for training, and the electroencephalogram source localization is realized by the brainstorm toolbox based on matlab2019b.

[0079] Figure 1 is a brief flowchart of the entire algorithm, and the entire algorithm is divided into five modules, including sensor space electroencephalogram signal acquisition and extraction, electroencephalogram source imaging and reconstruction, extraction of significantly different frequency bands, construction of feature separation double-encoding autoencoder, construction of deep learning hybrid model and model evaluation. Figure 2 (a) is a head electroencephalogram acquisition device diagram, Figure 2 (b) is an electroencephalogram channel distribution diagram.

[0080] The experimental scheme of sensor space electroencephalogram signal acquisition is shown in Figure 3 Twenty-two neurologically healthy adults participated in the data collection. Due to technical problems, the data of two participants could not be used, and the data of the remaining 20 participants (9 males, 11 females, average age = 30.85; standard deviation = 10.98) were analyzed. The subjects were injected with 0.6 μg / ml (mildly anesthetized state) propofol anesthetic, 1.2 μg / ml (moderately anesthetized state) propofol anesthetic, and the electroencephalogram signals of the subjects in the awake state and after anesthesia recovery were collected simultaneously. The state of the subjects was determined by behavioral tests, and the easy-to-anesthetize population and the not-easy-to-anesthetize population were distinguished by analysis.

[0081] Based on the above sensor space electroencephalogram signal data set, the algorithm flowchart shown in Figure 1 The human state evaluation method based on a deep learning hybrid model of the application comprises the following steps:

[0082] Step one: obtaining sensor space multi-channel electroencephalogram signals in an anesthetic state and preprocessing the obtained electroencephalogram signals;

[0083] In the process of obtaining sensor space multi-channel electroencephalogram signals in step one, the collected sensor space multi-channel electroencephalogram signals include two segments of electroencephalogram signals in the awake state and the moderately anesthetized state, and the electroencephalogram signals in the moderately anesthetized state are used to distinguish whether the subject is an easy-to-anesthetize population. The sensor space multi-channel electroencephalogram signals in the awake state and the moderately anesthetized state are used for subsequent analysis and training, and the obtained sensor space multi-channel electroencephalogram signals are represented as Wherein, m represents the number of subjects, each subject collects Noc lead electroencephalogram signals (in the experiment, 128 channels of high-density EEG data are collected, and the data of 91 channels on the scalp surface are retained for further analysis, so the maximum value of Noc is 91), the unit of EEG measurement is microvolt (μV), the sampling frequency is F (unit: Hz), n represents whether the subject is an easy-to-anesthetize population (in the experiment, n = 1 indicates that the subject belongs to the easy-to-anesthetize population, and n = 0 indicates that the subject belongs to the not easy-to-anesthetize population), and m is the number of Noc lead electroencephalogram signals collected by the mth subject in the experiment, represents a real matrix, which has Noc rows and t columns, and t is the number of sampling points. Assuming that the sampling time is t t seconds and the sampling frequency is F, then t = t t × F.

[0084] The preprocessing of the sensor space electroencephalogram signals in step one includes band-pass filtering, epoch division, baseline correction, and removal of eye movement or muscle artifacts. In band-pass filtering, a 0.5-45 Hz band-pass filter is used for all channel electroencephalogram signals of the retained sensor space electroencephalogram signals In epoch division, all channel electroencephalogram signals are divided into 10-second long epochs, each epoch is numbered as epoch e , e is the epoch number, and the maximum value of e depends on the number of divided epochs, the maximum value is e max . The baseline correction of each electroencephalogram signal numbered as epoch e is performed relative to the average voltage of the entire electroencephalogram signal , and in the removal of eye movement or muscle artifacts, the channels and epochs of abnormal noise are identified by calculating their normalized variance, and then manually rejected or retained by visual inspection. The sensor space electroencephalogram data obtained after preprocessing is where T is 10 × F in size, i.e., the length of the divided epoch is 10 seconds.

[0085] Step two: electroencephalography source imaging and reconstruction, i.e., calculating the sensor space electroencephalogram signals to obtain their mapping in the electroencephalography source space; and calculating the activity distribution of each brain region according to the Desikan-Killiany Atlas brain partition to perform electroencephalography source reconstruction and generate source space electroencephalogram signals; the specific process is as follows:

[0086] (1) Solving the forward problem of EEG, i.e. establishing a head model that can explain how the activity of neurons in the cerebral cortex is projected to the scalp through the brain volume effect and the conduction of the skull. The main parts of the head model include the geometry of the brain and the conductive properties of the different tissues (gray matter, white matter, cerebrospinal fluid, skull, skin) through which the electrical activity propagates. The present invention uses Freesurfer average based on magnetic resonance imaging (MRI) as the anatomical brain template and generates a head model for each participant using the default parameters of the Boundary Element Method (BEM) in OpenMEEG. The mathematical model of the scalp EEG signal is established as follows:

[0087]

[0088] Where E represents the sensor EEG signal obtained after step one; A represents the lead field, which is a zero lead domain matrix that simulates the distribution of neuron cells in the cerebral cortex and the change of their post-synaptic potential; X represents the dipole moment of the cortical current dipole, which simulates the distribution of neuron cells in the cerebral cortex and the change of their post-synaptic potential (source activity); noise represents the measurement noise.

[0089] (2) The minimum norm estimation method is used to solve the inverse problem.

[0090]

[0091] In theory, the optimal solution of X will be the solution that minimizes the residual between the recorded scalp EEG and the estimated scalp EEG in the sense of L2 norm least squares. Considering that the number of dipoles is often larger than the number of electrodes, many kinds of dipole distribution can produce similar results on the scalp, so L2 regularization is used to further constrain the solution to find the solution X that minimizes the residual and energy in the source space. This additional constraint is represented by the second term in (2); the regularization parameter λ is obtained by using the L curve method to balance the influence of the two terms. In the case of L2 norm, the analytical solution of (2) obtains the optimal estimation of the source distribution X, as shown in (3).

[0092]

[0093] When W is just the unit matrix, this solution is called the minimum norm estimation (MNE), which is the mapping of the sensor space EEG signal to the EEG source space, A T represents the inverse matrix of the lead field.

[0094] In the step two, the activity distribution of each brain region is calculated according to the Desikan-Killiany Atlas brain partition, the brain electrical source reconstruction is carried out, and the source space brain electrical signal process is generated, in the process, the number of dipoles in the Desikan-Killiany Atlas brain partition is classified, the measurement values of the dipoles in the same time in the region are averaged, and the output of the whole brain partition is regarded as the output of the whole brain partition, so that the source distribution matrix of the single subject is obtained (i.e. the EEG signal mapped from the sensor space to the source space), N c representing the sensor brain electrical signal the number of channels mapped in the source space (i.e. the number of regions of interest after the Desikan-Killiany Atlas partition division); T represents the number of sampling points, and T is 10*F, i.e. the length of the divided period is 10 seconds.

[0095] Step three: in all time domains, the unbiased weighted phase lag index connection matrix of all people and all channels to different samples is calculated; the permutation test between the two groups of people is used to find out the frequency band with significant difference and the corresponding unbiased weighted phase lag index connection matrix;

[0096] In the step three, the unbiased weighted phase lag index (debiased weighted phase lag index, DWPLI) connection matrix, the specific process is: in order to analyze the frequency in segments, and also to enhance the sample number, for the different epoch of the brain electrical signals of two different rows in the step two, representing the different channels of the source space brain electrical signal e segmented signal of the lower sample and time-frequency analysis is carried out, first, the time-frequency decomposition of and is carried out, and the time bin length is set to 0.04 seconds, and 250 time bins are generated for the 10-second-long and , numbered as bin ft (the maximum ft value in the experiment is 250, and 250 time bins can be divided for each segmented sample of each subject), and the number of frequency bins is set to 512, numbered as bin ft (the number of frequency bins in the experiment is 512, i.e. the maximum fi value is 512), and the cross spectrum is obtained by multiplying the Fourier transform signals and wherein represents the complex conjugate of , and it is noted that the cross spectrum is calculated by using the same epoch of the signals x and y e ​the same bin ft The phase lag index (PLI) of the cross spectrum

[0097]

[0098] The imaginary part of the cross spectrum sign is the sign function, and E denotes the mean value of all bins ft

[0099] The real part of the cross spectrum

[0100] The absolute value of the real part of the cross spectrum

[0101] PLI has a positive bias, i.e. PLI is usually higher for small values of ft in the bins ft used for the calculation. The debiased phase lag index (dbPLI) is calculated as

[0102]

[0103] The indices e j and e k denote the time bin number of different bins ft The weighted phase lag index (WPLI) is less sensitive to noise than the phase lag index, and is defined as

[0104]

[0105] Similar to PLI, WPLI has a positive bias. The DWPLI can be calculated as

[0106]

[0107] The indices j and k in the equation denote the value of the bins ft The time bins are accumulated, and note that for the calculation of DWPLI, it is required that for each frequency bin fi the corresponding (DWPLI has already been calculated for all bins ft , so it only depends on f), the of different channels are combined to generate the matrix The elements in the matrix ​The EEG signals of the i-th channel and the j-th channel in a certain epoch e bin fi This frequency was calculated within the chamber. (each in the experiment) Contains N c ×N c (element ), n represents whether the subject is in the easily anesthetized group (in the experiment, n=1 means the subject belongs to the easily anesthetized group, n=0 means the subject belongs to the difficult-to-anesthetized group), and m is the number of the m-th subject.

[0108] In step three, a permutation test between the two groups of people is used to identify frequency bands with significant differences. Specifically, this involves first solving the bin for each frequency band. fi of The mean of a matrix needs to be calculated by binning matrices with the same values. fi Below Each element in Add and divide by N c ×(N c -1) Obtain any frequency bin of any subject segment sample. Each epoch e You can get 512. Value. (The same bin) fi Value Form a sequence (The order of elements in the sequence is not required, i.e., it is randomly generated. In this experiment, a total of 512 sequences were generated. For each subject e) max different)

[0109] For each Perform a permutation test, with the number of permutations being times 1000. Within each permutation, [the following is done]. Randomly shuffle the sequence, but do not shuffle the labels. Calculate the mean value of the susceptible population for anesthesia based on the indices of the randomly shuffled sequence. forward Simultaneously, the mean value is calculated by taking the values ​​of the less easily anesthetized population from the randomly shuffled sequence. backpro minus timei timei represents the minus obtained from the i-th permutation. timei , all minus timei The MINUS sequence is obtained by sorting the elements in ascending order (from smallest to largest). The MINUS sequence has a total of TIME elements (each MINUS sequence has 1000 elements in this experiment).

[0110] Set the confidence coefficient `trust` (0.05 in the experiment) and solve for the upper limit of the threshold. and threshold lower limit The calculation formula is as follows:

[0111]

[0112]

[0113] In the above formula, MINUS{floor[TIME×(trust / 2)]} means taking the element with index floor[TIME×(trust / 2)] in the MINUS sequence, where floor represents the floor function.

[0114] Bins belonging to the same category (with the same n value) and of the same frequency fi All (with the same value) Take the average to get (A total of 512 × 2 were obtained in the experiment) Divided into two groups of people, and 512 frequency warehouses, through... Obtain the Dec sequence, its elements Will and and When a comparison is performed, Greater than or less At that time, it was assumed that the number was bin fi The frequency bands corresponding to the frequency bins are those with significant differences. This invention extracts these significantly different frequency bands and selects those frequency bins that belong to the frequency bins with significant differences. The total number of selected frequency bins is [number missing]. Significantly different frequency bands can be extracted from EEG signals in both the conscious and moderately anesthetized states. The corresponding frequency bands extracted from the conscious EEG signals... matrix Significantly different frequency bands were obtained from the EEG signals extracted by the sensor under moderate anesthesia. The matrix is The `set` parameter is set to indicate the state of the acquired spatial EEG signals from the sensor. It has two values: `awake` and `moderate`, corresponding to awake and moderately anesthetized states, respectively. Therefore, the matrix can be transformed into... The selected frequency bands with significant differences in the experiment are as follows: for the EEG data of the awake state, the frequency bands with significant differences in the sensor space EEG signal are 8.30-10.74 Hz, and the frequency bands with significant differences in the source space EEG signal are 8.79-10.25 Hz; for the EEG data of the moderate anesthesia state, the frequency bands with significant differences in the sensor space EEG signal are 7.32-10.25 Hz and 21.48-44.92 Hz, and the frequency bands with significant differences in the source space EEG signal are 7.32-9.77 Hz, 11.72-14.16 Hz, and 23.93-32.23 Hz.

[0115] The step four is to train the feature disentanglement double-encoder convolutional autoencoder model based on the unbiased weighted phase delay index connection matrix of the frequency bands with significant differences, and the specific process is as follows:

[0116] (1) Data conversion. According to the input requirements of the encoder, since the unbiased weighted phase delay index connection matrix obtained in step three is a symmetric matrix with elements of 0 on the diagonal, the non-diagonal elements in the matrix are taken half to deform into a one-dimensional vector, and the obtained one-dimensional vector is (the number of rows of the obtained one-dimensional vector is 2278, and the number of columns is 1, and the elements are sequentially placed in the one-dimensional vector in row priority order, and each symmetric matrix can obtain a )

[0117] (2) Model establishment. The feature disentanglement double-encoder convolutional autoencoder model has two parallel encoders and one decoder, and the two encoders are respectively used to extract individual features and pattern features in the data. It is specified that the name of the encoder for extracting individual features is E p , and the name of the encoder for extracting pattern features is E s, the individual feature and the pattern feature are added and fused before the decoder, and the decoder reconstructs the fused feature into the original data, and the decoder is denoted by D. (The structure of the encoder comprises a first convolutional layer, a first batch normalization layer, a first average pooling layer, a second convolutional layer, a second batch normalization layer, and a second average pooling layer, there are 8 convolutional kernels in the first and second convolutional layers, the convolutional kernel size is 3*1, the step is 1, padding is performed to make the input and output sizes the same, the filter kernel size of the first and second average pooling layers is 2*1, the step is 2, padding is performed to make the input and output sizes the same, the structures of the two encoders are the same, the structure of the decoder comprises a first convolutional layer, a first batch normalization layer, a first upsampling layer, a second convolutional layer, a second batch normalization layer, a second upsampling layer, and a third convolutional layer, there are 8 convolutional kernels in the first and second convolutional layers, the convolutional kernel size is 3*1, the step is 1, padding is performed to make the input and output sizes the same, the upsampling degree of the first and second upsampling layers is 2, there are 2 convolutional kernels in the third convolutional layer, the convolutional kernel size is 1*1, the step is 1, and no padding is performed)

[0118] (3) Construct a loss function. The two encoders E p and E s are input simultaneously, respectively, to obtain two vectors, respectively, and The two vectors are of the same size, the extracted individual feature and the pattern feature are fused, the fused data is input into the decoder D, and the reconstructed data is obtained.

[0119]

[0120]

[0121] A cross loss function is also constructed, and the one-dimensional vectors generated by the matrices of different n values of people belonging to different classes are compared. The cross loss function calculation formula is:

[0122]

[0123] For another one-dimensional vector of the same class but different set states as , a triplet loss function is constructed, and the calculation formula is:

[0124]

[0125]

[0126] α is a fixed value, and the total loss Loss function is calculated as follows:

[0127] Loss = Loss m,n + C Loss + TS Loss (15)

[0128] The total loss Loss is propagated back to the gradient, and the model parameters are optimized, including the parameters of E p , E s and D.

[0129] Step five: connect the trained autoencoder model with the convolutional neural network model to form a deep learning hybrid model, train the model, and finally evaluate the model. The specific process includes:

[0130] (1) Construct a deep learning hybrid model:

[0131] As shown in Figure 4 , connect the E s trained in step four with the convolutional neural network model, that is, use E s (X m,n ) as the input of the convolutional neural network model. The convolutional neural network model is composed of a first convolutional layer, a first batch normalization layer, a first average pooling layer, a second convolutional layer, a second batch normalization layer, a second average pooling layer, a flattening layer, a first full connection layer, a third batch normalization layer, a second full connection layer, a fourth batch normalization layer, and a third full connection layer in sequence. (The first convolutional layer has 16 convolutional kernels with a size of 3*1 and a step of 1, and padding is performed to make the input and output sizes the same. The second convolutional layer has 32 convolutional kernels with a size of 3*1 and a step of 1, and padding is performed to make the input and output sizes the same. The first and second average pooling layers have a filter size of 2*1 and a step of 2, and padding is performed to make the input and output sizes the same)

[0132] (2) Train the model:

[0133] Use the corresponding matrix of the frequency band with significant differences in the wakeful state and the corresponding matrix of the frequency band with significant differences extracted from the sensor electroencephalogram signal in the moderate anesthesia state to obtain a one-dimensional vector through data transformation as the data for model training. The label only contains whether this one-dimensional vector belongs to the easy-to-anesthetize population.

[0134] (3) Model evaluation:

[0135] The application adopts the method of M-fold cross validation to evaluate a human state evaluation method based on a deep learning hybrid model, only uses corresponding matrix A one-dimensional vector corresponding to the above operation is tested, that is, only the classification effect of the sample in the awake state is observed. The above operation is repeated M times, and finally the average recognition accuracy is calculated to evaluate the performance of the proposed new algorithm.

[0136] The following table is the accuracy of the specific embodiment of the application obtained by six-fold cross-validation on the test set of thirteen subjects, and the accuracy obtained is

[0137] The accuracy of ten-fold cross-validation on the test set of twenty subjects

[0138]

Claims

1. A human state evaluation method based on a deep learning hybrid model, characterized in that, The method comprises the following steps: Step 1: acquiring sensor space multi-channel electroencephalogram signals under anesthesia and preprocessing the acquired electroencephalogram signals; Step 2: electroencephalography and reconstruction, that is, calculating the sensor space electroencephalogram signals to obtain their mapping in the electroencephalography space; and calculating the activity distribution of each brain region according to the Desikan-Killiany Atlas brain partition, performing electroencephalography reconstruction, and generating source space electroencephalogram signals; Step 3: in all time domains, calculating the unbiased weighted phase lag index connection matrix of all people and all channels to different samples; using permutation test between two groups of people, finding out the frequency bands with significant differences and the corresponding unbiased weighted phase lag index connection matrix; Step 4: based on the unbiased weighted phase lag index connection matrix of the frequency bands with significant differences, training a feature disentangled double-encoder convolutional autoencoder model; Step 5: connecting the trained autoencoder model and a convolutional neural network model to form a deep learning hybrid model, training the model, and finally evaluating the model; The step 5 process specifically comprises: (1) constructing a deep learning hybrid model: E s is connected with the convolutional neural network model, that is, E s (X m,n ) is taken as the input of the convolutional neural network model, and the convolutional neural network model is sequentially composed of a first convolutional layer, a first batch normalization layer, a first average pooling layer, a second convolutional layer, a second batch normalization layer, a second average pooling layer, a flattening layer, a first full connection layer, a third batch normalization layer, a second full connection layer, a fourth batch normalization layer and a third full connection layer; (2) training the model: Utilizing the corresponding matrices of the significantly different frequency bands in the awake state and the corresponding matrices of the significantly different frequency bands extracted from the sensor electroencephalogram signals in the moderate anesthesia state The one-dimensional vector obtained by data transformation is used as the data for model training, and the label only contains whether the one-dimensional vector belongs to the easy anesthesia population. (3) model evaluation: The method of M-fold cross-validation was used for evaluation, only using the corresponding frequency bands with significant differences in the awake state matrix The corresponding one-dimensional vectors were tested, that is, only the classification effect of the samples in the awake state was observed, and the above operation was repeated M times, and finally the average recognition accuracy was calculated. 2.The human state evaluation method based on a deep learning hybrid model according to claim 1, characterized in that, The sensor space multi-channel electroencephalogram signals collected in the step one include two segments of electroencephalogram signals in a wake state and a moderate anesthesia state, and the obtained sensor space multi-channel electroencephalogram signals are represented as Wherein, n represents whether the subject is a susceptible anesthesia population, and m is the number of the electroencephalogram signals of the Noc guide collected by the mth subject in the experiment, represents a real matrix, There are Noc rows and t columns in the matrix, t is the number of sampling points, and it is assumed that the sampling time is t t seconds, and the sampling frequency is F, so t = t t × F.

3. The human state evaluation method based on the deep learning hybrid model according to claim 1, characterized in that, The pre-processing of the acquired electroencephalogram signal in step one includes band-pass filtering, time period division, baseline correction, and removal of eye movement or muscle artifact. In the band-pass filtering, the retained sensor space electroencephalogram signal is filtered by a band-pass filter of 0.5-45 Hz. In the epoch division, all the channel electroencephalogram signals are divided into multiple equal-length epochs, and each epoch is numbered as epoch e , e is the epoch number, the maximum value of e depends on the number of divided epochs, and the maximum value is e max , each electroencephalogram signal numbered as epoch e is baseline corrected relative to the average voltage of the entire electroencephalogram signal ; in the removal of eye movement or muscle artifact, the channel and epoch with abnormal noise are identified by calculating the normalized variance of the electroencephalogram signal , and then manually rejected or retained by visual inspection, and the sensor space electroencephalogram data obtained after preprocessing is 4.The human state evaluation method based on a deep learning hybrid model according to claim 1, characterized in that, The specific process of electroencephalography and reconstruction in step 2 is as follows: (1) solving the electroencephalogram forward problem, that is, establishing a head model that can explain how the activity of neurons in the brain cortex is projected to the scalp through brain volume effect and skull conduction, using Freesurfer average based on magnetic resonance imaging (MRI) as the anatomical brain template, and generating a head model for each participant using the Brainstorms default parameters of the boundary element method (BEM) in OpenMEEG, and the mathematical model of the scalp electroencephalogram signal is established as follows: represents the sensor EEG signal obtained after step one; A represents a lead field, which is a zero- domain matrix that models the distribution of neuron cells in the cerebral cortex and their post-synaptic potential changes; X represents the dipole moment of the cortical current dipoles that model the distribution of neuron cells in the cerebral cortex and their post-synaptic potential changes (source activity); noise represents the measurement noise; (2) using the least norm estimation method to solve the inverse problem; L2 regularization is used to further constrain the solution to find the solution X that minimizes the residual and energy of the source space, and this additional constraint is represented by the second term in equation (2); the regularization parameter λ is obtained by using the L curve method to balance the influence of the two terms, and in the case of L2 norm, the analytical solution of equation (2) obtains the optimal estimation of the source distribution X, as shown in equation (3): When W is just the identity matrix, this solution known as Minimum-norm Estimation (MNE), is the mapping of the sensor-space electroencephalogram (EEG) signals onto the brain's source space, A T denotes the leadfield matrix. 5.The human state evaluation method based on a deep learning hybrid model according to claim 1, characterized in that, The step two is to calculate the activity distribution of each brain region according to the Desikan-Killiany Atlas brain partition, carry out the brain electrical source reconstruction, and generate the source space brain electrical signal, specifically: according to the Desikan-Killiany Atlas brain partition, the number of dipoles is classified, the measurement values of the dipoles in the region at the same time are averaged, and the output of the whole brain partition is regarded as the output, so as to obtain the source distribution matrix of a single subject EEG signals mapped from the sensor space to the source space, N c representing the sensor brain electrical signal the number of channels mapped in the source space, that is, the number of regions of interest after the Desikan-Killiany Atlas partition division; T represents the number of sampling points. 6.The human state evaluation method based on a deep learning hybrid model according to claim 1, characterized in that, The specific calculation process of the unbiased weighted phase lag index connection matrix in step 3 is as follows: x and y are The cross-spectrum between two different rows of electroencephalogram signals, representing different channels of source space electroencephalogram signals, is calculated for different epochs of the two different channels of electroencephalogram signals x and y e The lower sample segments are denoted as signals and and time-frequency analysis is performed, a time bin length is set, and a time bin number is generated, denoted as bin ft A frequency bin length is set, and a frequency bin is generated, denoted as bin fi The cross-spectrum is obtained by multiplying the Fourier transform signals and wherein represents the complex conjugate of The cross-spectrum is calculated using the same epoch of signals x and y e The segments with the same bin ft value correspond completely in the time scale, and the phase lag index (PLI) of different channel signals x and y is defined as follows:​ represents the imaginary part of the cross-spectrum sign is the sign function, E represents the mean over all bins ft represents the absolute value of represents the absolute value of​ Since PLI has a positive bias, the calculation formula of unbiased phase lag index is as follows: Index e j and e k representing different bins ft The time period number, weighted phase lag index (WPLI), is less sensitive to noise than the phase lag index and is defined as: Similar to PLI, WPLI also has a positive bias, and DWPLI is calculated as follows: j and k in the formula represent the value of bin ft The time bin will be accumulated. When calculating DWPLI, it is required to calculate each frequency bin fi The corresponding Combine different channels To generate Matrix The elements in the matrix Indicate The i-th channel EEG signal and the j-th channel EEG signal in a certain epoch e Bin in fi This frequency bin n represents whether the subject is a susceptible anesthetized population, and m is the number of the mth subject.

7. The human state evaluation method based on the deep learning hybrid model according to claim 1, characterized in that, In step three, a permutation test between the two groups of people is used to identify frequency bands with significant differences. Specifically, this involves first solving the bin for each frequency band. fi of The mean of the matrix, binning the values ​​of the same type. fi Below Each element in Add and divide by N c ×(N c -1), to obtain any frequency bin of any subject segment sample. Each epoch e Get multiple Value; will use the same bin fi Value Form a sequence For each sequence performs a permutation test with times permutations, and for each permutation, it takes the values of the susceptible population in the sequence at the indices of the randomized sequence, and computes the mean mean randomizes, does not randomize the labels, and takes the values of the susceptible population in the sequence at the indices of the randomized sequence, and computes the mean mean forward , and takes the values of the non-susceptible population in the sequence at the indices of the randomized sequence, and computes the mean mean backpro , and computes the difference minus timei , timei represents the minus obtained by the ith permutation timei , and computes the difference minus timei , and sorts all the minus in ascending order to obtain a MINUS sequence, the MINUS sequence has a total of TIME elements; A confidence coefficient trust is set, and an upper threshold is solved and a lower threshold The calculation formula is: In the above formula, MINUS{floor[TIME×(trust / 2)]} represents the element with index floor[TiME×(trust / 2)] in the MINUS sequence, and floor represents the floor function; all the frequency bins of the same category and the same frequency bin take average by get the Dec sequence, whose elements will with and comparison, when greater than or less than , it is considered that the frequency bin numbered bin fi corresponding to the frequency band is a significantly different frequency band, and the significantly different frequency band is extracted, and the frequency bin belonging to the significantly different frequency band is selected, and the total number of the selected frequency bin is The sensor electroencephalogram signal in the awake state and the sensor electroencephalogram signal in the moderate anesthesia state can be extracted to have a significantly different frequency band, and the corresponding matrix of the significantly different frequency band extracted from the sensor electroencephalogram signal in the moderate anesthesia state is The set parameter is set to the state of the collected sensor spatial electroencephalogram signal, and has two values of awake and moderate, respectively corresponding to the awake state data and the moderate anesthesia state data, so the matrix can be changed to ​ 8.The human state evaluation method based on a deep learning hybrid model according to claim 1, characterized in that, The specific process of step 4 is as follows: (1) Data conversion, according to the input requirements of the encoder, take half of the non-diagonal elements in the matrix to deform into a one-dimensional vector, and the obtained one-dimensional vector is (2) Establish a feature disintegration double-encoder convolutional autoencoder model, which has two parallel encoders and one decoder. The two encoders are used to extract individual features and pattern features in the data respectively. The encoder name for extracting individual features is E p , and the encoder name for extracting pattern features is E s . Before the decoder, the individual features and the pattern features are added and fused. The decoder reconstructs the fused features into the original data, and the decoder is represented by D. (3) Constructing loss function; inputting one-dimensional vector two encoders E p and Ex respectively to obtain two vectors respectively and The two vectors have the same size, and the extracted individual features and pattern features are fused to obtain reconstruction data The reconstruction data is used to compare with the original input data to calculate the reconstruction loss, and the calculation formula is: At the same time, the cross loss function is constructed, which does not belong to the same class of people, that is, n value is different The one-dimensional vector generated by the matrix is The cross loss function calculation formula is: For another one-dimensional vector of the same class but different set state For another one-dimensional vector of the same class but different set state The triplet loss function is constructed, and the calculation formula is: α is a fixed value, and the total loss Loss function calculation formula is as follows: Loss = Loss m,n + C Loss + TS Loss (15) The total loss Loss is backpropagated to optimize the model parameters, including the parameters of E p , E s , and D.

Images