Classification model training method and device for dynamic representation of hidden variables of brain function signals

Abstract

The embodiment of the invention provides a classification model training method and device for dynamic representation of hidden variables of brain function signals. The method comprises the following steps: preprocessing a collected first electroencephalogram signal to obtain a second electroencephalogram signal, and carrying out hidden variable dynamic representation learning on the second electroencephalogram signal to obtain hidden variable dynamic representation corresponding to the second electroencephalogram signal. And decoding the hidden variable dynamic representation to obtain a first reduced electroencephalogram signal, and determining an electroencephalogram signal classification result according to the hidden variable dynamic representation, so that loss can be calculated at least according to the first reduced electroencephalogram signal, the electroencephalogram signal classification result and the hidden variable dynamic representation, and parameters of a classification model are trained according to the training loss. Therefore, the trained model can directly output the electroencephalogram signal classification result on the basis of only receiving the collected electroencephalogram signals, and classification of the electroencephalogram signals is realized by mining unobservable hidden variable dynamic representation in the electroencephalogram signals, so that the accuracy of emotion recognition is improved.

Description

Technical Field

[0001] The present application relates to the technical field of brain function signal classification, and in particular to a classification model training method and device for dynamic representation of latent variables of brain function signals. Background Art

[0002] EEG signals essentially reflect the spontaneous discharge behavior of neuronal group activities. They are driven by brain neural activities and are not easily controlled by subjective consciousness. Therefore, EEG signals can be used as an objective basis for emotion classification. Some medical devices can collect EEG signals from subjects and analyze them based on static modeling to obtain the emotions represented by EEG signals. However, this static modeling method makes it difficult to explore the characteristics of EEG signals in the time dimension, resulting in low accuracy in emotion recognition.

[0003] In some studies, the subjects' brains can also be regarded as dynamic systems, and the neural activity generated by the brain can be driven by unobservable hidden variables. In this way, the dynamic functional changes of the brain can be modeled through the dynamic evolution of hidden variables. For example, the short-term dependency hypothesis in resting dynamics (the state change at the current moment depends only on the state at the previous moment) and the state mutual exclusion hypothesis (only a single hidden state is active at each moment) are used to model and analyze the emotions corresponding to EEG signals.

[0004] However, the generation of emotions related to mental disorders, such as depression, stems from the disturbance of the brain's dynamic balance, which will become further disturbed over time. Therefore, heuristic data analysis methods such as the short-term dependence hypothesis and the state mutual exclusion hypothesis still have difficulty in improving the recognition accuracy of mental disorder emotions such as depression due to the lack of modeling of the data generation process, and lack the interpretability of mental disorder emotions such as depression. Summary of the invention

[0005] Some embodiments of the present application provide a classification model training method and device for dynamic representation of latent variables of brain function signals, so as to solve the problem of poor accuracy and interpretability in identifying emotions such as depression and other mental disorders based on EEG signals.

[0006] In a first aspect, some embodiments of the present application provide a classification model training method for dynamic representation of latent variables of brain function signals, the method comprising:

[0007] Collecting a first EEG signal; the first EEG signal is an EEG signal generated by an EEG signal source of the subject in a resting state;

[0008] Preprocessing the first EEG signal to obtain a second EEG signal; the second EEG signal is a signal obtained by mapping an EEG signal source corresponding to the first EEG signal to a standard brain template space;

[0009] Performing latent variable dynamic representation learning on the second EEG signal to obtain a latent variable dynamic representation of the second EEG signal;

[0010] Determining an EEG signal classification result according to the latent variable dynamic representation; and decoding the latent variable dynamic representation to obtain a first restored EEG signal;

[0011] The training loss is calculated at least according to the first restored EEG signal, the EEG signal classification result and the latent variable dynamic characterization, so as to optimize the network parameters of the latent variable dynamic characterization network based on the training loss until the training loss is no greater than a preset loss value, thereby obtaining an optimal latent variable dynamic characterization network.

[0012] In some feasible embodiments, the step of performing preprocessing on the first EEG signal to obtain the second EEG signal includes:

[0013] Performing notch filtering on the first EEG signal to obtain a first EEG signal in a target frequency band;

[0014] removing artifact signals in the first EEG signal located in the target frequency band;

[0015] Based on the trigger and delay of the resting-state experimental paradigm, extracting the eye-closing trial as a data segment in the first EEG signal from which the artifact signal is removed, so as to determine the primary preprocessing EEG signal based on the extracted data segment of the eye-closing trial;

[0016] The primary preprocessed EEG signal is mapped to a standard brain template space to obtain a second EEG signal.

[0017] In some feasible embodiments, the step of mapping the primary preprocessed EEG signal to a standard brain template space to obtain a second EEG signal comprises:

[0018] According to the preset head model structure, the head model is constructed using the anatomical structure image of the subject;

[0019] According to the sensor coordinate file of the EEG cap, the sensor position is calibrated and the sensor position is aligned to the surface of the head model to achieve spatial mapping of the EEG signal recording position;

[0020] Performing inverse problem solving based on the forward model to obtain the location result of the EEG signal source;

[0021] The positioning result of the EEG signal source is mapped to the standard brain template space to generate the spatial distribution of the EEG signal source of the primary preprocessed EEG signal to obtain the second EEG signal.

[0022] In some feasible embodiments, the step of performing latent variable dynamic representation learning on the second EEG signal to obtain a latent variable dynamic representation of the second EEG signal includes:

[0023] Acquiring the number of brain regions and time points of the second EEG signal;

[0024] Constructing a second-order tensor of the second electroencephalogram signal based on the number of brain regions and the number of time points;

[0025] A third-order tensor of the second EEG signal is constructed based on the second-order tensor of the second EEG signal, the EEG signal classification result, and the number of subjects corresponding to the EEG signal classification result.

[0026] In some feasible embodiments, the step of performing latent variable dynamic representation learning on the second EEG signal to obtain a latent variable dynamic representation of the second EEG signal further includes:

[0027] Calculate a variational posterior mean vector and a variational posterior covariance based on a multivariate Gaussian distribution of a third-order tensor of the second EEG signal;

[0028] Reparameterized sampling is performed on the variational posterior mean vector and the variational posterior covariance to obtain a latent variable dynamic representation corresponding to the second EEG signal.

[0029] In some feasible embodiments, the step of decoding the latent variable dynamic representation to obtain a first restored EEG signal includes:

[0030] Inputting the latent variable dynamic representation into a long short-term memory network to obtain latent variable space model distribution parameters based on the long short-term memory network decoding; the latent variable space model distribution parameters include a restored mean vector and a restored covariance based on a multivariate Gaussian distribution;

[0031] The mixing coefficient is calculated based on the mixing coefficient calculation expression and the hidden variable dynamic representation; the mixing coefficient calculation expression is:

[0032]

[0033] Among them, α jt ∈[0,1] and represents the softmax function, τ is the mixing degree hyperparameter, and the size of τ determines the activity level of each dimension of the latent variable at time t;

[0034] A linear summation is performed on the latent variable space model distribution parameters and the mixing coefficient to obtain a first restored EEG signal.

[0035] In some feasible embodiments, the step of calculating the loss function based on at least the first restored EEG signal, the EEG signal classification result and the latent variable dynamic representation includes:

[0036] The training loss is calculated based on the classification loss corresponding to the EEG signal classification result, the reconstruction error corresponding to the latent variable dynamic representation, and the Monte Carlo estimation result corresponding to the latent variable space model distribution parameter; the classification loss is calculated based on the true label value and the predicted label value of the EEG signal; the reconstruction error is obtained based on the predicted time point and the true time point corresponding to the latent variable dynamic representation during the decoding process; the Monte Carlo estimation result is calculated based on the preset class center parameter.

[0037] In some feasible embodiments, the step of determining the EEG signal classification result according to the latent variable dynamic representation includes:

[0038] The latent variable dynamic representation is input into the classification network in the latent variable dynamic representation network to obtain the EEG signal classification result output by the classification network according to the latent variable dynamic representation; the classification network includes a first convolutional layer, a dropout layer, a second convolutional layer, an average pooling layer, a fully connected layer, and an output layer connected in sequence; the number of convolution kernels of the first convolutional layer is 32, and the size of the convolution kernel is 3; the number of convolution kernels of the second convolutional layer is 16, and the size of the convolution kernel is 3; the fully connected layer includes 1 Dense layer containing 64 neurons; the output layer includes 1 Dense layer containing 1 neuron.

[0039] In some feasible embodiments, the step of collecting the first EEG signal includes:

[0040] An EEG cap of corresponding size is selected according to the subject's head circumference; the EEG cap is provided with a sensor for collecting EEG signals;

[0041] Setting the sampling frequency of the sensor;

[0042] The sensor collects a first electroencephalogram signal generated by the subject during the process of alternating eyes opening and eyes closing.

[0043] In a second aspect, the present application provides a classification model training device for dynamic representation of latent variables of brain function signals, comprising: a controller and an EEG cap including a sensor; the EEG cap is communicatively connected to the controller;

[0044] The EEG cap is configured to:

[0045] Collecting a first EEG signal; the first EEG signal is an EEG signal generated by an EEG signal source of the subject in a resting state;

[0046] The controller is configured to:

[0047] Preprocessing the first EEG signal to obtain a second EEG signal; the second EEG signal is a signal obtained by mapping an EEG signal source corresponding to the first EEG signal to a standard brain template space;

[0048] Performing latent variable dynamic representation learning on the second EEG signal to obtain a latent variable dynamic representation of the second EEG signal;

[0049] Determining an EEG signal classification result according to the latent variable dynamic representation; and decoding the latent variable dynamic representation to obtain a first restored EEG signal;

[0050] The training loss is calculated at least according to the first restored EEG signal, the EEG signal classification result and the latent variable dynamic characterization, so as to optimize the network parameters of the latent variable dynamic characterization network based on the training loss until the training loss is no greater than a preset loss value, thereby obtaining an optimal latent variable dynamic characterization network.

[0051] It can be seen from the above technical content that the embodiment of the present application provides a classification model training method and device for the dynamic representation of latent variables of brain function signals. The method obtains a second EEG signal by preprocessing the collected first EEG signal, and obtains the latent variable dynamic representation corresponding to the second EEG signal by learning the latent variable dynamic representation of the second EEG signal. Then, the first restored EEG signal is obtained by decoding the latent variable dynamic representation, and the EEG signal classification result is determined according to the latent variable dynamic representation, and then the loss can be calculated at least according to the first restored EEG signal, the EEG signal classification result and the latent variable dynamic representation, so as to train the parameters of the classification model according to the training loss. The model trained in this way can directly output the EEG signal classification result based on only receiving the collected EEG signal, and realize the classification of EEG signals by mining the unobservable latent variable dynamic representation in the EEG signal, so as to improve the accuracy of emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 A classification model training flow chart provided for some embodiments of the present application;

[0054] Figure 2 A schematic diagram of the composition of a classification model provided for some embodiments of the present application;

[0055] Figure 3 A schematic diagram of modeling the EEG signal generation process provided in some embodiments of the present application. DETAILED DESCRIPTION

[0056] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0057] In some emotion recognition scenarios, the brain can be regarded as a dynamic system, and the neural activity generated in the brain can be regarded as driven by unobservable hidden variables, and then the dynamic functional changes of the complex brain can be modeled based on the dynamic evolution of hidden variables. In some embodiments, modeling can be based on the short-term dependency hypothesis (the state change at the current moment depends only on the state at the previous moment) and the state mutual exclusion hypothesis (only a single hidden state is active at each moment). However, brain function signals have temporal and nonlinear characteristics, and the generation mechanism of brain function signals is a random process with high uncertainty. However, these two assumptions can only be modeled by a set of observed model parameters at a given moment, which is not suitable for the characteristics of brain function signals. Therefore, it is necessary to adopt a more flexible modeling method with multiple states to model brain dynamics to solve the problems of low accuracy and low interpretability in identifying subjects' emotions based on EEG signals.

[0058] like Figure 1 As shown, in order to solve the above problems, in some embodiments, a classification model training method for dynamic representation of latent variables of brain function signals is provided, and the steps include:

[0059] S100: Collecting a first electroencephalogram signal.

[0060] In some embodiments, it is necessary to collect EEG signals as training data. Among them, the first EEG signal can be an EEG signal generated by the EEG signal source of the subject in a resting state. The resting state here can refer to the process in which the subject is in an alternating state of opening and closing eyes. In this process, EEG signals can be collected by an EEG cap (sensor in the EEG cap) arranged at a preset position of the subject.

[0061] In some embodiments, the step of collecting the first EEG signal includes:

[0062] An EEG cap of corresponding size is selected according to the head circumference of the EEG signal provider; a sensor for collecting EEG signals is provided in the EEG cap.

[0063] Sets the sampling frequency of the sensor.

[0064] The sensor collects the first EEG signal generated by the EEG signal provider during the process of alternating eyes opening and eyes closing.

[0065] It is understandable that in the process of selecting an EEG cap, it is necessary to select it according to the subject's head circumference, so that the position of the sensor in the EEG cap and the EEG signal source is more closely matched, the data acquisition accuracy is improved, and the modeling accuracy of the head model is improved. In addition, the EEG signal acquisition system to which the EEG cap belongs can include 128 leads (the number of leads can be selected according to actual needs), and the reference electrode can be set at a fixed point on the top of the head. Before acquisition, the impedance of each electrode needs to be checked to ensure that the electrode is in good contact. For example, the electrode impedance can be used if it is below 40kΩ.

[0066] During the test preparation process, the sampling frequency of the sensor needs to be set in advance to control the sensor to collect the EEG signal generated by the subject according to the sampling frequency. For example, the sampling frequency of the sensor can be set to 500 Hz.

[0067] In some embodiments, after the EEG cap is placed on the subject's head, the EEG signal generated by the subject during the process of alternating eyes opening and closing can be continuously collected within 30 minutes, that is, the first EEG signal. The first EEG signal can be understood as the EEG signal generated by the subject in a resting state.

[0068] S200: Preprocessing the first EEG signal to obtain a second EEG signal.

[0069] It is understandable that before the collected first EEG signal is input into the classification model, the first EEG signal needs to be preprocessed so that the classification model can extract the dynamic representation of latent variables in the first EEG signal. Among them, the preprocessing method used for the first EEG signal can include filtering, artifact removal, trial extraction, and source tracing. In this way, some EEG signal data with excessive noise or obvious data can be removed through preprocessing, thereby improving the data quality during training or reasoning.

[0070] In some embodiments, the step of performing preprocessing on the first EEG signal to obtain the second EEG signal includes:

[0071] Perform notch filtering on the first EEG signal to obtain a first EEG signal in a target frequency band.

[0072] Remove the artifact signal in the first EEG signal located in the target frequency band.

[0073] Based on the trigger and delay of the resting-state experimental paradigm, the eye-closing trial is extracted as a data segment in the first EEG signal from which the artifact signal is removed, so as to determine the primary preprocessing EEG signal based on the extracted data segment of the eye-closing trial.

[0074] The primary preprocessed EEG signal is mapped to a standard brain template space to obtain a second EEG signal.

[0075] In some embodiments, the first EEG signal may be firstly imported into a system with a preprocessing function, such as MATLAB. The specific preprocessing system used is not limited here, and the purpose is to improve the data quality of the first EEG signal through multiple preprocessing methods.

[0076] It should be noted that in the preprocessing process, the data for training or reasoning in the first EEG signal can be first screened according to the pre-set triggers and delays. The trigger here refers to a specific signal corresponding to a specific node of the EEG signal, for example, corresponding to the moment of stimulus presentation. The trigger can be used to synchronize experimental events and EEG signals. The delay can refer to the time difference between when the trigger is triggered and when the potential change associated with the event is observed in the EEG signal. The required data can be extracted from the first EEG signal by relying on the trigger and the delay.

[0077] It is understandable that triggers can be used to mark events. For example, triggers can be used to mark eye closing events or eye opening events. In this way, time series data corresponding to eye closing events can be extracted as needed through triggers and delays.

[0078] In addition, the first EEG signal can be filtered by setting the filter interval and the filter threshold to obtain the first EEG signal in the target frequency band. For example, the first EEG signal is subjected to a 0.1-70 Hz bandpass filter and a 50 Hz notch filter. This can improve the data accuracy of the first EEG signal, reduce redundant data, and help improve the accuracy and efficiency of model training.

[0079] After obtaining the first EEG signal in the target frequency band, an artifact signal removal operation can be performed on the first EEG signal in the target frequency band. For example, a multiple artifact removal algorithm is used to remove the artifact signal in the first EEG signal to obtain a valid first EEG signal. Furthermore, based on the trigger and delay of the preset resting state experimental paradigm, the eye closure trial can be extracted as a primary preprocessed EEG signal.

[0080] It should be noted that when the subjects close their eyes, the alpha waves in their EEG signals will be enhanced, and the amplitude of high-frequency signals such as beta waves will decrease, so the spectral characteristics of the EEG signals will shift toward the low-frequency direction, making it easier to learn and identify the activity patterns of the subjects' brains, which is conducive to extracting the dynamic representation of the latent variables corresponding to the EEG signals. In addition, extracting the eye-closing trials as the primary preprocessing EEG signals can also filter out bad data to facilitate model training and reasoning.

[0081] In addition, the trigger and delay of the resting state experimental paradigm here have the same function as the trigger and delay in the above embodiment. According to the trigger and delay corresponding to the closed eyes state, the EEG signal corresponding to the subject in the rhinitis state can be intercepted from the effective first EEG signal, that is, the primary pre-processed EEG signal. For example, the closed eyes EEG signals of 5 trials can be extracted from the first EEG signal collected from the subject, and the EEG signals of the 5 trials can be averaged to obtain a smooth primary pre-processed EEG signal.

[0082] In this way, the primary preprocessed EEG signal can be further mapped to the standard brain template space to obtain the spatial distribution of the EEG signal source of the EEG signal, thereby tracing the primary preprocessed EEG signal, that is, obtaining a second EEG signal after tracing.

[0083] In some embodiments, the step of mapping the primary preprocessed EEG signal to the standard brain template space to obtain the second EEG signal comprises:

[0084] The head model is constructed using the subject's head anatomical structure image.

[0085] According to the sensor coordinate file of the EEG cap, the sensor position is calibrated and the sensor position is aligned to the surface of the head model to achieve spatial mapping of the EEG signal recording position.

[0086] An inverse problem is solved based on the forward model to obtain the positioning result of the EEG signal source.

[0087] The positioning result of the EEG signal source is mapped to the standard brain template space to generate the spatial distribution of the EEG signal source of the primary preprocessed EEG signal to obtain the second EEG signal.

[0088] It is understandable that there will be some differences in the head models of different subjects, so a head model corresponding to the subject can be constructed based on the head anatomical structure image of the subject. And based on the position information of the sensor in the EEG cap, the sensor position can be calibrated and registered on the surface of the head model, thereby realizing the spatial mapping of the EEG signal recording position. This can improve the accuracy of data acquisition and is conducive to determining the location of the EEG signal source corresponding to a specific emotion in the subsequent analysis of EEG signals. Taking depression as an example, spatial mapping can help trace the source of the EEG signal of depression.

[0089] After determining the spatial mapping of the EEG signal recording position, the inverse problem can be solved according to the forward model to obtain the positioning result of the EEG signal source. It should be noted that the forward model can be used to describe the mathematical model of the relationship between the EEG signal source and the EEG signal recorded on the subject's scalp. The forward model takes into account factors such as the current source distribution inside the brain, the geometry of the head, and the conductivity characteristics of different tissues, and can predict the potential distribution of each electrode position on the scalp through calculation or simulation methods. Furthermore, based on the potential distribution, the inverse problem can be solved, that is, the position and intensity of the EEG signal source inside the brain can be inferred based on the potential distribution results.

[0090] Among them, the algorithm for solving the inverse problem can adopt the sLORETA (low-resolution brain electromagnetic tomography) algorithm. By using the potential distribution provided by the forward model combined with the EEG signal or the brain magnetic signal to optimize the search for the distribution of the EEG signal source, the error between the simulated potential distribution and the actual measurement data can be reduced.

[0091] After locating the EEG signal sources of different subjects, the EEG signal sources can be mapped to the standard brain template space, thereby ensuring that the EEG data generated by different subjects have a unified standard and comparability during the training or reasoning process of the classification model, so as to eliminate the impact of individual differences on the research results and improve the reliability and repeatability of the research. In this way, after mapping the EEG signal source to the standard brain template space, a second EEG signal that has been traced can be obtained. When the model is trained or model reasoned based on the second EEG signal, the corresponding relationship between the classification result and the EEG signal source can be fully combined.

[0092] S300: Perform latent variable dynamic representation learning on the second EEG signal to obtain a latent variable dynamic representation of the second EEG signal.

[0093] In some embodiments, after obtaining the second EEG signal, the second EEG signal can be input into a classification model for dynamic representation of latent variables of brain function signals. Figure 2 As shown, the classification model includes an inference model network, a generation model network, and a classification network. The inference model network is used to generate a latent variable dynamic representation based on the second EEG signal. The generation model network is used to restore the restored EEG signal based on the latent variable dynamic representation output by the inference model network, so as to calculate the training loss based on the restored EEG signal, and then optimize the classification model through the training loss. The classification network is used to generate an EEG signal classification result based on the latent variable dynamic representation.

[0094] It can be understood that after obtaining the second EEG signal, the second EEG signal can be input into the inference model network, and the inference model network outputs the dynamic representation of the latent variables.

[0095] Among them, the inference model network can be a bidirectional recurrent neural network, such as a long short-term memory network. The inference model network can be constructed by a normalization layer, an activation layer, a dropout layer, and a regularization layer. Based on the recursive connection and time-dependent characteristics, the hidden variable state associated with the time series can be dynamically updated, and then the mean vector and covariance matrix of the multivariate Gaussian distribution of the EEG signal source can be fully mined in the process of hidden variable dynamic representation learning. The hidden variable dynamic representation corresponding to the EEG signal can be generated through the mean vector and covariance matrix.

[0096] It should be noted that the data input to the inference model network is actually the third-order tensor of the second EEG signal. That is, the steps of performing latent variable dynamic representation learning on the second EEG signal to obtain the latent variable dynamic representation of the second EEG signal include:

[0097] The number of brain regions and time points of the second EEG signal are obtained.

[0098] A second-order tensor of the second electroencephalogram signal is constructed based on the number of brain regions and the number of time points.

[0099] A third-order tensor of the second EEG signal is constructed based on the second-order tensor of the second EEG signal, the EEG signal classification result, and the number of subjects corresponding to the EEG signal classification result.

[0100] In some embodiments, the second-order tensor of the second EEG signal can be determined by first obtaining the number of brain regions and time points corresponding to the second EEG signal. The second-order tensor can be expressed as Taking the case where the number of brain regions of the subject is 100 and the time series length is 20000 as an example, the second-order tensor of the second EEG signal can be expressed as The second-order tensor here is equivalent to a two-dimensional matrix consisting of the number of EEG signal time points and the number of brain regions of the subject. Among them, the second EEG signal at each time t can be regarded as a matrix with a time-varying mean vector m t and the covariance matrix C t is generated by a multivariate Gaussian distribution process, that is, X t =N(m t ,C t ).

[0101] Furthermore, in order to achieve the purpose of training, the dimensions corresponding to the number of subjects and the number of classification result categories can be added to the second-order tensor to obtain a third-order tensor Where N k is the number of subjects in this category, K is the number of categories corresponding to the classification results, represents the preprocessed EEG signal time series of the k1 category subject. If the classification result corresponds to the mental emotion / emotional disease detection result, K is the number of emotion / emotional disease categories. This allows adding labels during the training process to facilitate the inference model network to learn the output results.

[0102] In this way, the third-order tensor of the second EEG signal can be input into the inference model network, and the inference model network outputs the latent variable dynamic representation of the second EEG signal.

[0103] It can be understood that the process of determining the dynamic representation of latent variables by the inference model network is the process of mining the mean vector and covariance matrix of the multivariate Gaussian distribution corresponding to the second EEG signal at each moment. That is, performing latent variable dynamic representation learning on the second EEG signal, and obtaining the step of latent variable dynamic representation of the second EEG signal also includes:

[0104] A variational posterior mean vector and a variational posterior covariance are calculated based on the multivariate Gaussian distribution of the third-order tensor of the second EEG signal.

[0105] Reparameterized sampling is performed on the variational posterior mean vector and the variational posterior covariance to obtain a latent variable dynamic representation corresponding to the second EEG signal.

[0106] In some embodiments, the latent variable dynamic representation can be expressed as Where J is a multivariate Gaussian distribution state. Different latent variable dynamic representations can correspond to different emotion recognition results / emotional disease recognition results. That is, subjects with different emotional diseases have different latent variable dynamic representations of brain neural activity. Based on the representation method of latent variable dynamic representation, in the case of multiple subjects, the latent variable dynamic representation can be recorded as and then, is the representation of the jth hidden state of the k1 category subject at time point t, and represents the latent variable representation of the k1 category subject at the tth time point. That is, at time t, the neural activity interaction pattern of the brain of this type of subject is in the closed-eye resting state. The latent state representation of any category The calculation process can be expressed as: in and are the variational posterior mean and covariance of the multivariate Gaussian distribution, respectively. The variational posterior covariance is a diagonal matrix. Given an EEG signal, the inference model network outputs the parameters of the variational posterior distribution, i.e., the variational posterior mean and covariance of the multivariate Gaussian distribution, through the following expression:

[0107]

[0108] Furthermore, we can use the reparameterization technique to sample from the variational posterior distribution inference, specifically by sampling from a standard normal distribution Its calculation expression is:

[0109]

[0110] where I is the identity matrix, ε s represents the sth sampling from the normal distribution N(0,I), is a Square roots of the diagonal elements.

[0111] It should be noted that in the above calculation process, because it is necessary to calculate the time-dependent z t Perform Bayesian inference, so that z t The estimates of z will have large uncertainties, which will propagate to z over time. t However, the mean μ of the latent variable j and covariance D j is a global parameter whose inference contains the dynamic evolution information at all time points. Therefore, μ is learned by using a point estimation method with a trainable static scaling factor j and D j .

[0112] Among them, the static scaling factor can be 1 / batch size × number of batches. In this way, by introducing a static scaling factor, the certainty of the dynamic representation of latent variables can be improved, thereby improving the accuracy of the classification results and the training effect of the model. Among them, the batch size and the number of batches can be set according to demand. In some embodiments, the batch size can be 64, and the number of batches in each batch cycle can be 100, so that the static scaling factor is 1 / 6400. Then, when the model is updated with gradients and the training loss is calculated, it can be ensured that the gradients calculated from different batches have a consistent scale when updating the model parameters, so as to avoid instability in model training due to excessive differences in gradient scales.

[0113] S400: determining an EEG signal classification result according to the latent variable dynamic representation; and decoding the latent variable dynamic representation to obtain a first restored EEG signal.

[0114] It is understandable that the inference model network can input the dynamic representation network into the classification network, and the classification network outputs the emotion recognition result according to the latent variable dynamic representation. The step of determining the EEG signal classification result according to the latent variable dynamic representation includes:

[0115] The latent variable dynamic representation is input into the classification network in the latent variable dynamic representation network to obtain the EEG signal classification result output by the classification network according to the latent variable dynamic representation; the classification network includes a first convolutional layer, a dropout layer, a second convolutional layer, an average pooling layer, a fully connected layer, and an output layer connected in sequence; the number of convolution kernels of the first convolutional layer is 32, and the size of the convolution kernel is 3; the number of convolution kernels of the second convolutional layer is 16, and the size of the convolution kernel is 3; the fully connected layer includes 1 Dense layer containing 64 neurons; the output layer includes 1 Dense layer containing 1 neuron.

[0116] In some embodiments, the classification network is designed based on a temporal convolutional network, which aims to extract local and high-level features of time series data through a multi-layer network, and ultimately complete the binary classification task. Taking the training process of the classification model as an example, when the dynamic representation of the latent variable is input into the classification network, the classification network can extract local features from the time dimension through the first convolutional layer, and introduce nonlinear changes through the ReLU activation function, and apply L2 regularization to limit the model weight value to reduce the risk of overfitting. Furthermore, 30% of the neurons are randomly discarded during the training process through the Dropout layer connected to the post-order position of the first convolutional layer to improve the generalization ability of the classification network. Then, higher-level time series features are further extracted through the second convolutional layer, and the ReLU activation function and L2 regularization are used.

[0117] Through the learning of two convolutional layers, the classification network can gradually capture the complex patterns and feature relationships in the time series corresponding to the dynamic representation of latent variables. Then, by introducing a global average pooling layer, the time dimension of each feature channel is averaged and the time dimension information is compressed into a vector of fixed size. After further nonlinear learning of the extracted feature relationship through the fully connected layer, the classification result is output based on the output layer. Among them, the activation function used in the output layer can be Sigmoid.

[0118] In this way, the classification network can fully extract the key features in the time series corresponding to the latent variable dynamic representation, and then perform the task of emotion classification based on the latent variable dynamic representation. It can be understood that if we take the mental illness identification process as an example, the classification task performed by the classification network during the retraining process is actually a binary classification task (the latent variable dynamic representation can correspond to the mental illness patients / healthy control group subjects).

[0119] In addition, in order to train a complete classification model, in some embodiments, the first restored EEG signal is obtained by decoding the latent variable dynamic representation through the generative model network, and the first restored EEG signal is used as an important component for calculating the training loss. The generative model network can be a long short-term memory network, that is, a recurrent neural network, and the generative model network can be set as a unidirectional recurrent network. The step of decoding the latent variable dynamic representation to obtain the first restored EEG signal includes:

[0120] The latent variable dynamic representation is input into a long short-term memory network to obtain latent variable space model distribution parameters based on the long short-term memory network decoding; the latent variable space model distribution parameters include a restored mean vector and a restored covariance based on a multivariate Gaussian distribution.

[0121] The mixing coefficient is calculated based on the mixing coefficient calculation expression and the hidden variable dynamic representation; the mixing coefficient calculation expression is:

[0122]

[0123] Among them, α jt ∈[0,1] and represents the softmax function, τ is the mixing degree hyperparameter, and the size of τ determines the activity level of each dimension of the latent variable at time t.

[0124] A linear summation is performed on the latent variable space model distribution parameters and the mixing coefficient to obtain a first restored EEG signal.

[0125] like Figure 3 As shown in the figure, a schematic diagram of the modeling process of EEG signal generation is provided, taking the classification results of the classification network for identifying the mental disorders of the subjects as an example, which includes EEG signals reflecting changes in brain functional activity Dynamic representation of latent variables in brain neural activity Category K of mental disorders.

[0126] It is understandable that there is a causal dependency between latent variable dynamic representation and mental disorders, and different mental disorders correspond to different latent variable dynamic representations. In addition, latent variable dynamic representation has an impact on the generation process of EEG signals. The embodiment of the present application aims to train a classification model by mining the characteristics of latent variable dynamic representation of EEG signals, so that the classification model can learn the correspondence between latent variable dynamic representation and mental disorders. Therefore, latent variable dynamic representation can be regarded as the main factor in determining whether the subject has a mental disorder.

[0127] It should be noted that the latent variable space model corresponding to the dynamic representation of latent variables corresponds to the static spatial distribution of the functional connection of brain neural activity, and the functional connection of a specific latent variable can be captured by the correlation between brain regions in the covariance. However, the distribution of the latent variable space model is a static distribution. Therefore, in the process of decoding the dynamic representation of latent variables, the generative network also needs to introduce a mixing coefficient for characterizing the dynamic changes of EEG signals. Among them, the mixing coefficient can be recorded as α t . Mixing coefficient α t It also provides a low-dimensional and interpretable dynamic representation of time-series data such as EEG signals.

[0128] It can be seen that in the decoding process of the generative model network, the latent variable dynamic representation can be regarded as a mixture coefficient and the latent variable space model distribution parameter. The latent variable space model distribution parameter is the mean μ j and covariance D j .

[0129] In this way, the expression of the EEG signal generation process based on the dynamic representation of latent variables can be determined as:

[0130] P(X,Z,K)=P(X|Z,K)P(Z,K)=P(X|Z)P(Z|K)p(K)

[0131] Among them, the category k of each sample obeys the classification distribution, that is, and The probability that each sample belongs to one of the K categories is π. The latent variable representation of any category at each time point is The calculation expression is: in represents the dynamic representation of latent variables before time point t-1, and are the mean vector and covariance matrix of the multivariate Gaussian distribution, respectively.

[0132] In this way, based on the above derivation, the generative model network can predict the dynamic representation of the latent variables at the next time point when the dynamic representation of the latent variables at the past time point is known. Input to the generative model network:

[0133]

[0134] in and represents the affine transformation in the recurrent neural network, ξ() is a function used to preserve the standard deviation is a positive softplus function, and LSTM is a long short-term memory network.

[0135] In this way, the generative model network can obtain the covariance and mean in the spatial distribution parameters of the latent variables.

[0136] In some embodiments, the mixing coefficient α t Can be represented by latent variables The transformation calculation shows that the calculation expression of each state j of the hidden variable at time point t is:

[0137]

[0138] Among them, α jt ∈[0,1] and Represents the softmax function, τ is a hyperparameter called the degree of mixing, and the size of τ determines the activity of each dimension of the latent variable at the current moment.

[0139] In this way, the second EEG signal X obeys the multivariate Gaussian distribution t Parameters, instant variable mean m t and covariance C t , through the α t and the parameters of the spatial static distribution of the observable EEG signal (mean μ j and covariance D j ) can be obtained by linear addition. The linear addition expression is:

[0140]

[0141] It can be understood that in some embodiments of the present application, the generation model network module 140 can first sample the latent vector time series from the latent variable dynamic representation learned by the inference model network, use the latent variable dynamic representation based on the past time points in the unidirectional recurrent neural network to predict the latent variable dynamic representation at the next time point, perform a nonlinear transformation on the latent variable dynamic representation at one time point to obtain the mixing coefficient, and linearly add the mean vector and covariance matrix of the static distribution of the EEG signal space and the dynamic mixing coefficients of different categories of latent variable time series to fit the latent variable dynamic representation of the EEG signal in a more interpretable descriptive form, which can be used to calculate the training loss and improve the generalization ability of the neural network model.

[0142] S500: Calculate the training loss at least according to the first restored EEG signal, the EEG signal classification result and the latent variable dynamic characterization, so as to optimize the network parameters of the latent variable dynamic characterization network based on the training loss until the training loss is no greater than a preset loss value, thereby obtaining an optimal latent variable dynamic characterization network.

[0143] It is understandable that the parameters of the classification model in the initial state are all random, so it is necessary to optimize the parameters of the model through training to obtain the optimal classification model. That is, the step of calculating the loss function based on at least the first restored EEG signal, the EEG signal classification result and the latent variable dynamic representation includes:

[0144] The training loss is calculated based on the classification loss corresponding to the EEG signal classification result, the reconstruction error corresponding to the latent variable dynamic representation, and the Monte Carlo estimation result corresponding to the latent variable space model distribution parameter; the classification loss is calculated based on the true label value and the predicted label value of the EEG signal; the reconstruction error is obtained based on the predicted time point and the true time point corresponding to the latent variable dynamic representation during the decoding process; the Monte Carlo estimation result is calculated based on the preset class center parameter.

[0145] In some embodiments, the classification network can not only output classification results based on the dynamic representation of latent variables output by the inference model network, but also calculate the training loss based on key data such as the classification results, the first EEG restoration signal, and the dynamic representation of latent variables, and then reversely optimize the various parameters of the classification model through the training loss.

[0146] Among them, the classification network needs to constrain the distribution of latent variables when calculating the training loss to obtain the maximized variational boundary, that is, to minimize the variational free energy F. In other words, minimizing the variational free energy F can be used as the target loss. In order to distinguish the distribution of different categories of latent variables, label information can be introduced. The calculation formula for minimizing the variational free energy F is:

[0147]

[0148] Among them, q(z,k|x) and p(z,k,x) can represent the probability distribution of the inference model and the generation model about the data x, category k, and hidden variable z respectively.

[0149] Based on the data x generation process, we can obtain: p(x,z,k)=p(x|z,k)p(k,z)=p(x|z)p(z|k)p(k) and q(z,k|x)=q(z|x)q(k|x).

[0150] Therefore, the variational free energy is converted to F = -E q(z,k|x) log[p(x|z)p(z|k)p(k)]+E q(z,k|x) logq(z,k|x).

[0151] In some embodiments, the classification target of the classification network is still to distinguish between subjects with mental disorders and healthy subjects as an example. Assuming that there are only two types of people in the data, subjects with mental disorders (depression) and healthy subjects, the number of states J in the multivariate Gaussian distribution of the latent variable can be 8. Among them, the number of states J can be selected according to actual conditions, for example, in the range of 6 to 12. In this way, under the premise of introducing labels during the classification model training process, the classification loss that can be calculated by the classification network is:

[0152]

[0153] where k k and are the true label value and the label value predicted by the model, and k is the category index.

[0154] The final loss function is as follows:

[0155]

[0156] In the formula is the reconstruction error between the predicted time point and the true value in the computational generation model, where Indicates that only one sample is used when Monte Carlo estimation is used, k is the unique hot encoding of category k, μ jt and σ jt are the mean vector and covariance matrix of the latent variable time series learned by the inference model network, μ kj,t It is the class center of the mean of the time series of different categories of brain latent variables. It is set as a trainable parameter with an initial value of 0 in the model and is updated with the optimization iteration of the neural network model.

[0157] In this way, during the training process, the classification network can calculate the training loss based on key data such as the dynamic representation of latent variables output by the inference model network, the first restored EEG signal generated by the generation model network, and the second EEG signal corresponding to the first restored EEG signal, and further optimize the various parameters in the classification model through the training loss.

[0158] In some embodiments, the training loss is the above-mentioned loss function constructed based on minimizing variational free energy. The training loss is not greater than the preset loss value, which may mean that the loss function constructed based on minimizing variational free energy is in a convergence state during the training process, that is, the difference between the loss function calculated last time and the loss function calculated currently is less than or equal to the preset difference preset range.

[0159] It should be noted that after the optimal classification model is obtained through training, the EEG signal of the subject is input into the optimal classification model during the use of the optimal classification model. The optimal classification model will be classified by the classification network according to the dynamic representation of the latent variables output by the inference model network based on the EEG signal. In this process, the classification network does not need to calculate the training loss, and the generation model network does not need to decode the dynamic representation of the latent variables output by the inference model network.

[0160] In some embodiments, after optimizing the parameters of the classification model, the performance of the classification model can also be evaluated. For example, the classification model can be evaluated by using a 5-fold cross-validation method with 50% of the subjects left out. Or in the process of optimizing the parameters of the classification model, the classification model can also be evaluated by using a 5-fold cross-validation method with 50% of the subjects left out. The implementation process is, for example: the data set of the tested data is divided into 5 parts, and 4 of them are used as training data and 1 is used as test data for testing in turn.

[0161] In some embodiments, all samples of 50 depressed subjects and 50 healthy subjects can be divided into 5 parts according to the subjects, that is, all samples of 50 depressed subjects and 50 healthy subjects can be divided into 5 parts according to the individuals, and the number of depressed subjects and healthy subjects in each part is as equal as possible. During each training and testing, 1 part of the data is reserved for performance evaluation, and the remaining 4 parts of the data are used to train the neural network model. This process is performed a total of 5 times according to the different reserved data. During training, the Adam optimizer can be used to update the parameters in the neural network model.

[0162] For example, the initial value of the learning rate is 0.0002, the batch size is set to 64, the reconstruction loss and KL loss are weighed through the KL (Kullback-Leibler Divergence) annealing strategy, and the early stopping method is used to save the optimal model parameters; after completing the test on each piece of data, the accuracy, sensitivity, specificity, and F1 score are calculated, and then the average of the 5 results is calculated as the final performance evaluation of the neural network model.

[0163] In some embodiments, a classification model training device for dynamic representation of latent variables of brain function signals is also provided. The classification model training device includes a controller and an EEG cap including a sensor. The EEG cap is communicatively connected to the controller. The EEG cap is configured to: collect a first EEG signal; the first EEG signal is an EEG signal generated by an EEG signal source of a subject in a resting state.

[0164] The controller is configured as:

[0165] The first EEG signal is preprocessed to obtain a second EEG signal. The second EEG signal is a signal obtained by mapping an EEG signal source corresponding to the first EEG signal to a standard brain template space.

[0166] Perform latent variable dynamic representation learning on the second EEG signal to obtain a latent variable dynamic representation of the second EEG signal.

[0167] Determining an EEG signal classification result according to the latent variable dynamic representation; and decoding the latent variable dynamic representation to obtain a first restored EEG signal.

[0168] The training loss is calculated at least according to the first restored EEG signal, the EEG signal classification result and the latent variable dynamic characterization, so as to optimize the network parameters of the latent variable dynamic characterization network based on the training loss until the training loss is no greater than a preset loss value, thereby obtaining an optimal latent variable dynamic characterization network.

[0169] In some embodiments, after training, the brain function signal latent variable dynamic representation classification model training device can be used as a mature reasoning model in medical devices used for emotion recognition and mental disorder identification.

[0170] The embodiment of the present application provides a classification model training method and device for latent variable dynamic representation of brain function signals. The method obtains a second EEG signal by preprocessing the collected first EEG signal, and obtains the latent variable dynamic representation corresponding to the second EEG signal by learning the latent variable dynamic representation of the second EEG signal. Then, the first restored EEG signal is obtained by decoding the latent variable dynamic representation, and the EEG signal classification result is determined according to the latent variable dynamic representation, and then the loss can be calculated at least according to the first restored EEG signal, the EEG signal classification result and the latent variable dynamic representation, so as to train the parameters of the classification model according to the training loss. The model trained in this way can directly output the EEG signal classification result based on only receiving the collected EEG signal, and realize the classification of EEG signals by mining the unobservable latent variable dynamic representation in the EEG signal, so as to improve the accuracy of emotion recognition.

[0171] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A classification model training method for dynamic representation of latent variables of brain function signals, characterized in that: include: collecting a first EEG signal; The first EEG signal is an EEG signal generated by the EEG signal source of the subject in a resting state; Performing preprocessing on the first EEG signal to obtain a second EEG signal; The second EEG signal is a signal obtained by mapping an EEG signal source corresponding to the first EEG signal to a standard brain template space; Performing latent variable dynamic representation learning on the second EEG signal to obtain a latent variable dynamic representation of the second EEG signal; Determining the EEG signal classification result according to the latent variable dynamic representation; And, decoding the latent variable dynamic representation to obtain a first restored EEG signal; The training loss is calculated at least according to the first restored EEG signal, the EEG signal classification result and the latent variable dynamic characterization, so as to optimize the network parameters of the latent variable dynamic characterization network based on the training loss until the training loss is no greater than a preset loss value, thereby obtaining an optimal classification model.

2. The method according to claim 1, characterized in that The step of performing preprocessing on the first EEG signal to obtain a second EEG signal comprises: Performing notch filtering on the first EEG signal to obtain a first EEG signal in a target frequency band; removing artifact signals in the first EEG signal located in the target frequency band; Based on the trigger and delay of the resting-state experimental paradigm, extracting the eye-closing trial as a data segment in the first EEG signal from which the artifact signal is removed, so as to determine the primary preprocessing EEG signal based on the extracted data segment of the eye-closing trial; The primary preprocessed EEG signal is mapped to a standard brain template space to obtain a second EEG signal.

3. The method according to claim 2, characterized in that The step of mapping the primary preprocessed EEG signal to a standard brain template space to obtain a second EEG signal comprises: According to the preset head model structure, the head model is constructed using the anatomical structure image of the subject; According to the sensor coordinate file of the EEG cap, the sensor position is calibrated and the sensor position is aligned to the surface of the head model to achieve spatial mapping of the EEG signal recording position; Performing inverse problem solving based on the forward model to obtain the location result of the EEG signal source; The positioning result of the EEG signal source is mapped to the standard brain template space to generate the spatial distribution of the EEG signal source of the primary preprocessed EEG signal to obtain the second EEG signal.

4. The method according to claim 3, characterized in that The step of performing latent variable dynamic representation learning on the second EEG signal to obtain the latent variable dynamic representation of the second EEG signal comprises: Acquiring the number of brain regions and time points of the second EEG signal; Constructing a second-order tensor of the second electroencephalogram signal based on the number of brain regions and the number of time points; A third-order tensor of the second EEG signal is constructed based on the second-order tensor of the second EEG signal, the EEG signal classification result, and the number of subjects corresponding to the EEG signal classification result.

5. The method according to claim 4, characterized in that The step of performing latent variable dynamic representation learning on the second EEG signal to obtain the latent variable dynamic representation of the second EEG signal further includes: Calculating a variational posterior mean vector and a variational posterior covariance based on a multivariate Gaussian distribution of a third-order tensor of the second EEG signal; Reparameterized sampling is performed on the variational posterior mean vector and the variational posterior covariance to obtain a latent variable dynamic representation corresponding to the second EEG signal.

6. The method according to claim 1, characterized in that The step of decoding the latent variable dynamic representation to obtain a first restored EEG signal comprises: Inputting the latent variable dynamic representation into a long short-term memory network to obtain latent variable space model distribution parameters based on the long short-term memory network decoding; the latent variable space model distribution parameters include a restored mean vector and a restored covariance based on a multivariate Gaussian distribution; The mixing coefficient is calculated based on the mixing coefficient calculation expression and the hidden variable dynamic representation; the mixing coefficient calculation expression is: Among them, α jt ∈[0,1] and represents the softmax function, τ is the mixing degree hyperparameter, and the size of τ determines the activity level of each dimension of the latent variable at time t; A linear summation is performed on the latent variable space model distribution parameters and the mixing coefficient to obtain a first restored EEG signal.

7. The method according to claim 1, characterized in that The step of calculating the loss function at least according to the first restored EEG signal, the EEG signal classification result and the latent variable dynamic representation comprises: The training loss is calculated based on the classification loss corresponding to the EEG signal classification result, the reconstruction error corresponding to the latent variable dynamic representation, and the Monte Carlo estimation result corresponding to the latent variable space model distribution parameter; the classification loss is calculated based on the true label value and the predicted label value of the EEG signal; the reconstruction error is obtained based on the predicted time point and the true time point corresponding to the latent variable dynamic representation during the decoding process; the Monte Carlo estimation result is calculated based on the preset class center parameter.

8. The method according to claim 1, characterized in that: The step of determining the EEG signal classification result according to the latent variable dynamic representation comprises: The latent variable dynamic representation is input into the classification network in the latent variable dynamic representation network to obtain the EEG signal classification result output by the classification network according to the latent variable dynamic representation; the classification network includes a first convolutional layer, a dropout layer, a second convolutional layer, an average pooling layer, a fully connected layer, and an output layer connected in sequence; the number of convolution kernels of the first convolutional layer is 32, and the size of the convolution kernel is 3; the number of convolution kernels of the second convolutional layer is 16, and the size of the convolution kernel is 3; the fully connected layer includes 1 Dense layer containing 64 neurons; the output layer includes 1 Dense layer containing 1 neuron.

9. The method according to claim 1, characterized in that: The step of collecting the first electroencephalogram signal comprises: An EEG cap of corresponding size is selected according to the subject's head circumference; the EEG cap is provided with a sensor for collecting EEG signals; Setting the sampling frequency of the sensor; The sensor collects a first electroencephalogram signal generated by the subject during the process of alternating eyes opening and eyes closing.

10. A classification model training device for dynamic representation of latent variables of brain function signals, characterized in that: include: A controller and an EEG cap including a sensor; the EEG cap is in communication connection with the controller; The EEG cap is configured to: collecting a first EEG signal; The first EEG signal is an EEG signal generated by the EEG signal source of the subject in a resting state; The controller is configured to: Performing preprocessing on the first EEG signal to obtain a second EEG signal; The second EEG signal is a signal obtained by mapping an EEG signal source corresponding to the first EEG signal to a standard brain template space; Performing latent variable dynamic representation learning on the second EEG signal to obtain a latent variable dynamic representation of the second EEG signal; Determining the EEG signal classification result according to the latent variable dynamic representation; And, decoding the latent variable dynamic representation to obtain a first restored EEG signal; The training loss is calculated at least according to the first restored EEG signal, the EEG signal classification result and the latent variable dynamic characterization, so as to optimize the network parameters of the latent variable dynamic characterization network based on the training loss until the training loss is no greater than a preset loss value, thereby obtaining an optimal classification model.

Images