Electroencephalogram depression recognition system and method based on knowledge distillation and meta transfer learning

Through knowledge distillation and metatransfer learning methods, the knowledge of 128 channels of EEG signals was extracted and fine-tuned in the student model, which solved the problem of poor generalization performance of model in EEG signal depression recognition, and improved the accuracy and adaptability of depression recognition .

CN118986349APending Publication Date: 2024-11-22BEIJING INST OF TECH
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202411044372.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art has poor generalization performance in the recognition of depression of EEG signals, which is difficult to adapt to different data sets and subjects, and cross-domain differences lead to inaccurate identification results.

Method used

Using a method based on knowledge distillation and meta-transfer learning, knowledge is extracted from 128 channels of EEG signals through the teacher model and transferred it to the student model. Combined with meta-learning, the student model is fine-tuned to adapt to the 3-channel EEG signal data set. , improve the accuracy of depression recognition.

Benefits of technology

Effectively refining the knowledge of multi-channel EEG signals to the depression recognition model with few-channel EEG signals improves the performance of depression recognition and enhances the recognition performance of the model on different distributions and subjects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118986349A_ABST
    Figure CN118986349A_ABST
Patent Text Reader

Abstract

The invention provides an electroencephalogram depression recognition system based on knowledge distillation and meta transfer learning. The electroencephalogram depression recognition system is an electroencephalogram depression recognition model training framework composed of a plurality of models. The teacher model extracts knowledge from the 128-channel electroencephalogram signal, and the student model learns hidden knowledge from the teacher model, continues to perform fine tuning adaptation on the 3-channel electroencephalogram and tests the depression recognition effect; according to the framework, the knowledge of the multi-channel electroencephalogram signals can be effectively extracted into the depression recognition model of the few-channel electroencephalogram signals, so that the depression recognition performance of the depression recognition model of the few-channel electroencephalogram signals is improved; a training thought of meta learning is applied after an online knowledge distillation process, a student model is regarded as a meta model, and a part of data on a new data set is utilized to carry out fine tuning on the meta model to obtain a final fine tuning model, so that the depression recognition performance of the model on three-channel electroencephalogram signals under different distribution, different subjects and different conditions is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of electroencephalogram (EEG) signal processing, and in particular relates to an EEG depression recognition system and method based on knowledge distillation and meta-transfer learning. Background Art

[0002] Depression is a mental illness characterized by emotional and cognitive dysfunction, which can lead to suicide in severe cases. According to statistics from the World Health Organization, 3.8% of the world's population suffers from depression, of which 5% is among adults. The clinical diagnosis and treatment rates of depression are very low. Therefore, early identification of depression is of great significance for its effective treatment. Electroencephalogram (EEG) signals can objectively reflect the internal state of the human brain and can effectively and quickly diagnose and identify clinical depression.

[0003] Since subjective factors can affect the identification of depressive disorders to varying degrees, it is very important to find reliable objective indicators and establish identification methods and models. More and more scholars have begun to use EEG signals to find biomarker features that can predict the onset of depressive disorders and establish machine learning models in order to identify depressive disorders more accurately and timely. The machine recognition models in this regard mainly include logistic regression classifiers (LR), support vector machines (SVM), random forests (RF), multilayer perceptrons (MP), etc., and have achieved certain research results.

[0004] In recent years, deep learning, as a method that uses multi-layer neural networks and optimization techniques to automatically extract useful information from data, can help doctors diagnose and predict diseases accurately and quickly. The development of deep learning technology has provided a new solution for machine recognition of depressive disorders. Many researchers have used convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and graph neural networks (GNN) to analyze the degree of depression reflected by EEG signals.

[0005] However, traditional machine learning and deep learning tasks require a large number of labeled training samples with the same distribution as the test samples. But in reality, collecting enough training samples requires a lot of manpower and material costs, and the EEG signals vary greatly between individuals. It is difficult for the trained model to achieve good adaptation results on new subjects. Transfer learning is a very promising machine learning method to solve the above problems. It aims to extract information from one or more source tasks and then apply it to the target task, making full use of multiple types of training samples with different configurations to improve the performance of the model in the target field.

[0006] In the depression recognition scenario based on EEG signals, to obtain an accurate neural network model, large-scale data training is required. However, due to factors such as acquisition equipment, task paradigm, and differences between subjects, there are large differences in EEG signals between different data sets or different subjects in the same data set, making it difficult to directly use EEG signals for depression recognition across devices / paradigms / subjects. Therefore, how to minimize the differences between data in different fields or find common features among them is an unresolved problem. Summary of the invention

[0007] In view of this, the present invention proposes an EEG depression recognition method and system based on knowledge distillation and meta-transfer learning to alleviate the problem of poor model generalization performance in the prior art. By learning from public EEG data sets, the common features in the distribution of EEG data are learned, thereby training a pre-trained model that can quickly adapt to similar data sets.

[0008] An EEG depression recognition system based on knowledge distillation and meta-transfer learning, characterized by comprising a data acquisition module, a data preprocessing module, a knowledge distillation module, a meta-transfer learning module, and a depression recognition module;

[0009] The data acquisition module acquires multiple resting-state EEG data sets, including: a first data set, a second data set, and a third data set, wherein the first data set uses 128-channel resting-state EEG signals of depression and normal subjects in the MODMA data set, the second data set uses 3-channel resting-state EEG signals of depression and normal subjects in the MODMA data set of the same subjects as in the first data set, and the third data set uses another 3-channel resting-state EEG signal data set of depression and normal subjects collected from more subjects;

[0010] The data preprocessing module is used to perform denoising and feature extraction on resting-state EEG signal data;

[0011] The knowledge distillation module is provided with a teacher model and a student model; the teacher model is trained using the first data set, and during the training process, the knowledge extracted from the 128-channel EEG signal is fed to the student model;

[0012] The student model is inputted with the data of the second data set, and receives the knowledge fed by the teacher model to learn the common knowledge of the 128-channel EEG signals in the first data set and the 3-channel EEG signals in the second data set, and the unique knowledge of the 3-channel EEG signals in the second data set;

[0013] The meta-transfer learning module uses the resting-state EEG signal data of the third data set to fine-tune the student model;

[0014] The depression recognition module is used to implement a depression recognition task for the resting-state EEG signal data to be recognized based on the fine-tuned student model.

[0015] 2. An EEG depression recognition system based on knowledge distillation and meta-transfer learning as described in claim 1, characterized in that the student model is based on an improved EEGNet neural network structure; and the teacher model is based on an improved GCN neural network structure.

[0016] 3. The EEG depression recognition system based on knowledge distillation and meta-transfer learning according to claim 2, characterized in that the teacher model comprises two series-connected graph convolution modules GCN and a fully connected layer and a softmax layer connected in sequence thereafter;

[0017] The student model includes four two-dimensional convolutional layers connected in series, and a sub-classifier module consisting of a bottleneck layer, a fully connected layer, and a SoftMax layer connected in sequence is added after each two-dimensional convolutional layer; each of the sub-classifier modules outputs a set of probability prediction values;

[0018] The teacher model and the student model are trained simultaneously, and the loss used in the training update process is as follows:

[0019] The first loss is used to calculate the loss between the teacher model's predicted label and the actual data label;

[0020] The second loss is used to calculate the loss between the labels predicted by each sub-classifier module of the student model and the actual data labels;

[0021] The third loss is used to calculate the loss between the probability distribution predicted by the SoftMax layer of each sub-classifier module of the student model and the probability distribution predicted by the SoftMax layer of the teacher model;

[0022] The fourth is used to calculate the loss between the data feature distribution after the fully connected layer of each sub-classifier module of the student model and the data feature distribution after the fully connected layer of the teacher model.

[0023] The present invention has the following beneficial effects:

[0024] The present invention proposes an EEG depression recognition system based on knowledge distillation and meta-transfer learning, which is an EEG depression recognition model training framework composed of several models; the teacher model extracts knowledge from 128-channel EEG signals, and the student model learns hidden knowledge from the teacher model and continues to fine-tune and adapt on 3-channel EEG and test the depression recognition effect; the framework of the present invention can effectively extract the knowledge of multi-channel EEG signals into the depression recognition model of few-channel EEG signals to improve the latter's depression recognition performance.

[0025] In order to enable the student model to achieve better depression recognition effect on the collected 3-channel EEG signal dataset, the present invention also applies the training idea of ​​meta-learning after the online knowledge distillation process, regards the student model as a meta-model, and uses a part of the data on the new dataset to fine-tune the meta-model to obtain the final fine-tuning model, thereby improving the model's depression recognition performance for 3-channel EEG signals with different distributions, different subjects, and different situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of an EEG depression recognition system based on knowledge distillation and meta-transfer learning provided in an embodiment of the present invention.

[0027] Figure 2 A data preprocessing flowchart generally applicable to different EEG signal data sets is provided in an embodiment of the present invention.

[0028] Figure 3 A flowchart of a method for identifying EEG depression based on knowledge distillation and meta-transfer learning provided in an embodiment of the present invention.

[0029] Figure 4 An architecture diagram of a teacher model and a student model based on a knowledge distillation method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0031] The nerve cells in the brain communicate with each other through some weak neural signals, producing some measurable changes, which can be called brain waves. In one implementation, these changes can be recorded by electroencephalogram (EEG). Exemplarily, EEG can be a reflection of brain activity and state, which depicts the physiological patterns of humans in different tasks or different psychological states. Therefore, EEG can be used to record the changes in brain activity waves, and EEG signals can be used to identify depressive states.

[0032] In the process of depression identification through EEG signals, different EEG signals are often required to have the same characteristic distribution. However, due to the influence of differences in different acquisition environments, different experimental paradigms, different acquisition equipment, different sources of subjects, etc., different EEG signals will have large distribution differences. Therefore, the cross-domain distribution differences of EEG signals lead to inaccurate depression identification results. In addition, the distribution differences of EEG signals will cause the depression identification methods across subjects or paradigms to be unable to be applied to depression identification methods of other subjects or paradigms. When there are identification tasks for new subjects, a large number of labeled EEG signals are required, which will also lead to a large workload and low recognition efficiency of emotional states.

[0033] The present invention combines two deep learning model training strategies of transfer learning methods, including a meta-learning method and a knowledge distillation method. Meta-learning is a transfer learning method, which is usually understood as "learning to learn", and refers to the process of improving the learning algorithm in multiple learning stages, including a meta-training stage and a meta-testing stage. In the meta-training process, the internal learning algorithm solves the task defined by the data set and the target, and the external algorithm updates the internal learning algorithm so that the model it learns improves the external target. In the meta-testing process, the meta-model obtained based on the meta-training is fine-tuned on the new data set, and a fine-tuned model is quickly converged to perform the corresponding task on the new data set. Therefore, the core idea of ​​meta-learning is to train a model on various learning tasks (meta-training stage) so that it can quickly adapt to new tasks (meta-testing stage) with only a few examples. Knowledge distillation is an effective deep neural network compression and acceleration technology, which uses a trained teacher model to provide knowledge, and the student model obtains the teacher's knowledge through distillation training, and transfers the knowledge of the complex teacher model to the simple student model at the cost of slight performance loss.

[0034] In a first aspect, an embodiment of the present invention provides an EEG depression recognition method based on knowledge distillation and meta-transfer learning, comprising: obtaining multiple EEG data sets collected in a resting state, including public data sets and some collected data sets; based on each set of data in the multiple EEG data sets, using them at different stages of training to establish a corresponding EEG depression recognition meta-model; further, based on each set of data in the multiple EEG data sets, performing denoising and normalization processing on the EEG data to obtain preprocessed EEG data; performing feature value extraction on the preprocessed EEG data to obtain EEG feature data; training a preset machine learning model based on the EEG feature data to obtain a depression recognition model based on EEG data, and giving a depression recognition result after simple adaptation on a new EEG data set based on the depression recognition model based on EEG data.

[0035] In some embodiments, a preprocessing step is performed on the EEG signal, including: selecting a reference electrode to re-reference the data to remove the baseline drift of the signal; using a high-pass filter with a cutoff frequency of 0.5 Hz and a low-pass filter with a cutoff frequency of 45 Hz to remove high and low frequency noise in the EEG signal; selecting an appropriate window size (such as 5s or 10s) to divide the data into segments for time analysis; using different feature extraction methods to extract features from the EEG data segments in the time domain, frequency domain, and time-frequency domain to obtain several columns of EEG feature information for training the depression recognition model based on EEG data.

[0036] In a second aspect, an embodiment of the present invention further provides an EEG depression recognition system based on knowledge distillation and meta-transfer learning, comprising a data acquisition module, a data preprocessing module, a model building module, a knowledge distillation module, a meta-transfer learning module, and a depression recognition module; wherein the data acquisition module is used to acquire multiple resting-state EEG data sets, including multiple EEG data sets with different channel settings; the data preprocessing module is used to perform feature extraction based on the multiple resting-state EEG data sets to obtain feature representation; the model building module is used to establish a corresponding EEG signal depression recognition model suitable for the required channel configuration based on the feature representation of the multiple groups of EEG data sets; the knowledge distillation module is used to transfer hidden knowledge from the teacher EEG depression recognition model to the student EEG depression recognition model based on the training strategy of knowledge distillation; the meta-transfer learning module is used to fine-tune the student model based on the student EEG depression recognition model and a part of the data of a new batch of data sets; the depression recognition module is used to implement the depression recognition task on another part of the data of a new batch of data sets based on the fine-tuned student EEG depression recognition model.

[0037] In order to solve the problem of inaccurate recognition results of depression caused by differences in the distribution of EEG signals, some embodiments of the present application provide an EEG depression recognition method based on knowledge distillation and meta-transfer learning. The method can learn the common features of EEG signals from different experimental paradigms, different acquisition equipment, and different sources of subjects as much as possible, and solve the problem of distribution differences of EEG signals caused by multiple factors in depression recognition. In order to facilitate the understanding of the technical solutions in some embodiments of the present application, each step is described in detail below in conjunction with some specific embodiments and drawings.

[0038] Figure 1It is a schematic diagram of an EEG depression recognition system based on knowledge distillation and meta-transfer learning provided by an embodiment of the present invention. The system includes a data acquisition module, a data preprocessing module, a model building module, a knowledge distillation module, a meta-transfer learning module, and a depression recognition module. In this embodiment, first, a plurality of resting EEG data sets are acquired through the data acquisition module, including a plurality of EEG data sets with different channel settings, for example, data set 1 uses the resting EEG signals of 128-channel depression and normal subjects in the MODMA data set, data set 2 uses the resting EEG signals of 3-channel depression and normal subjects in the MODMA data set of the same subjects as in data set 1, and data set 3 uses a resting EEG signal data set of 3-channel depression and normal subjects collected separately and containing more subjects.

[0039] The MODMA dataset is a multimodal open dataset for the analysis of mental disorders. The dataset includes EEG and audio data from clinically depressed patients and matched normal controls. The dataset includes not only data collected by the traditional 128-channel electrode cap, but also data collected by a new wearable 3-channel EEG collector. The 128-channel EEG signals of 53 subjects at rest and stimulation states and the 3-channel EEG signals of 55 subjects at rest were recorded;

[0040] The 128-channel EEG experiment included 53 participants, including 24 outpatients diagnosed with depression (13 males, 11 females, aged 16-56 years) and 29 healthy controls (20 males and 9 females; aged 18-55 years); the 3-channel EEG experiment included 55 participants, including 26 outpatients diagnosed with depression (15 males, 11 females, aged 16-56 years) and 29 healthy controls (19 males, 10 females, aged 18-55 years). In this example, the data of the experimental participants who participated in both experiments were taken and used to train the teacher model and the student model in the knowledge distillation module respectively.

[0041] Dataset 3 is a 3-channel EEG data collected from another group of subjects using an EEG device with the same channel configuration as Dataset 2. It is divided into two parts for input in the knowledge distillation module, one part is used to fine-tune the student EEG depression recognition model, and the other part is used to complete the EEG depression recognition task.

[0042] Subsequent use Figure 2The data preprocessing method shown is used for preprocessing and feature extraction; then, the data of multiple data sets are input into the corresponding models through the model building module, and different depression recognition models suitable for 128-channel and 3-channel EEG signal are established; then, the depression recognition model suitable for 3-channel EEG data of data set 3 is trained by combining meta-learning and knowledge distillation methods; the depression recognition module implements the depression recognition task on part of the data of data set 3 based on the trained depression recognition model suitable for 3-channel EEG data of data set 3.

[0043] Figure 2 It is a general data preprocessing flow chart for different EEG signal data sets provided by an embodiment of the present invention. The preprocessing process includes: selecting a reference electrode to re-reference the data to remove the baseline drift of the signal; using a high-pass filter with a cutoff frequency of 0.5Hz and a low-pass filter with a cutoff frequency of 45Hz to remove high and low frequency noise in the EEG signal; selecting a suitable window size (such as 5s or 10s) to divide the data segments for time analysis; using different feature extraction methods to extract features from EEG data segments in the time domain, frequency domain, and time-frequency domain, using bandpass filtering for the segmented EEG data, setting the corresponding cutoff frequency of the bandpass filtering, and obtaining the power spectral density, differential entropy features, differential asymmetry and other columns of EEG feature information of four different frequency bands of theta (4-8Hz), alpha (8-13Hz), beta (13-30Hz) and gamma (31-45Hz) to train the depression recognition model based on EEG data.

[0044] Figure 3It is a flowchart of an EEG depression recognition method based on knowledge distillation and meta-transfer learning provided by an embodiment of the present invention. First, a public EEG data set and a collected EEG signal are obtained, and then the EEG signal is preprocessed and the required features are extracted using the preprocessing method. In the model training stage, the goal of the present invention is to extract the common knowledge of EEG from the 128-channel EEG signal and transfer it to the depression recognition model for the 3-channel EEG through an online knowledge distillation process. The first step is to use the data set 1 based on the improved EEGNet (EEGNet is a neural network structure specially designed for processing electroencephalogram (EEG) data, combining the advantages of CNN and deep separable convolution, aiming to improve the efficiency and accuracy of EEG signal processing) to train a teacher model, and during the training process, the knowledge extracted from the 128-channel EEG signal is fed to the student model at the same time. The student model is based on the GNN neural network structure, and the input is the data of the data set 2, and receives the knowledge fed by the teacher model to learn the common knowledge of the 128-channel EEG signals in the data set 1 and the 3-channel EEG signals in the data set 2 and the unique knowledge of the 3-channel EEG in the data set 2. The second step is to use part of the collected data set to fine-tune the student model to adapt it to the characteristics of the new data set. The third step is to use the fine-tuned student model to implement the depression recognition task on another part of the collected data set.

[0045] Figure 4 It is a diagram of the architecture of a teacher model and a student model based on a knowledge distillation method provided by an embodiment of the present invention. The figure shows the specific architecture of the teacher model and the student model of the embodiment of the present invention and the loss calculation that the training and updating process depends on. The teacher model includes two graph convolutional modules (Graph Convolutional Network, GCN is a deep learning model for processing graph data, which aggregates and updates node features through graph convolution layers, and uses the information of graph structure to improve the effect of representation learning) connected in sequence, and then connected in sequence to the fully connected layer and softmax layer; the student model uses the improved EEGNet network structure, including 4 two-dimensional convolutional layers connected in series, and adds a group of bottleneck layers, fully connected layers, and SoftMax layers after each two-dimensional convolutional layer. The sub-classifier module is connected in sequence, and each outputs a set of probability prediction values. The main consideration for adding the bottleneck layer is to reduce the influence between each shallow classifier. In the training stage, all shallow parts with corresponding classifiers learn the knowledge of different parts of the teacher model.

[0046] In the experiment, we observe the knowledge acquisition from the shallow to the deep level of the student model and analyze the learning process that the student goes through in the process of integrating and transferring the teacher's knowledge. As mentioned earlier, the teacher model and the student model are trained simultaneously, and the dependent losses during the training update process are as follows:

[0047] Loss 1 is used to calculate the loss between the teacher model’s predicted labels and the actual data labels;

[0048] Loss 2 is used to calculate the loss between the labels predicted by each sub-classifier module of the student model and the actual data labels;

[0049] loss 3 is used to calculate the loss between the probability distribution predicted by the SoftMax layer of each sub-classifier module of the student model and the probability distribution predicted by the SoftMax layer of the teacher model;

[0050] Loss 4 is used to calculate the loss between the data feature distribution after the fully connected layer of each sub-classifier module of the student model and the data feature distribution after the fully connected layer of the teacher model.

[0051] In the meta-learning process, for each new dataset, the pre-trained model can be fine-tuned to adapt to the specific data. The fine-tuning process usually involves performing several batches of training on the new dataset, updating the model's weights through backpropagation and gradient descent algorithms. This meta-learning method is equivalent to combining the idea of ​​transfer learning, that is, the knowledge learned from one task (pre-training dataset) can be transferred to another related but different task (new dataset). The pre-trained model is used as a starting point, and fine-tuning can converge quickly on fewer training samples because the model already has good initial parameters and representation capabilities.

[0052] The pre-trained model has learned common features and patterns through large-scale datasets, which are useful for a variety of tasks. Through fine-tuning, the model learns the details and features of specific tasks on the new dataset, making the model better adapted to the new task. In this process, the generalization ability of the model is improved in two aspects:

[0053] Shared feature learning: The general features learned by the pre-trained model on large-scale data can be applied to new tasks, and the high-level representation of these features is preserved and adjusted during the fine-tuning process.

[0054] Task-specific fine-tuning: The fine-tuning process allows the model to adapt to the characteristics of the new dataset, thereby improving performance on the new task.

[0055] The student model obtained through the above training process is regarded as a pre-trained model in the meta-learning process. Then, a part of the collected data set 3 is used to fine-tune the model so that it can adapt to the new channel setting to obtain the corresponding fine-tuned model. Finally, another part of the collected data set 3 is input into the fine-tuned model for depression recognition to obtain the depression recognition result.

[0056] It can be seen from the above technical solutions that the EEG depression recognition system based on knowledge distillation and meta-transfer learning provided in the above embodiments can train the depression recognition model through online knowledge distillation and meta-learning. More common information between different channel configurations can be obtained through the depression recognition model. Therefore, the depression recognition model can more accurately distinguish the depressive state of the EEG signal and improve the accuracy of depression recognition.

[0057] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An EEG depression recognition system based on knowledge distillation and meta-transfer learning, characterized in that: It includes data acquisition module, data preprocessing module, knowledge distillation module, meta-transfer learning module, and depression recognition module; The data acquisition module acquires multiple resting-state EEG data sets, including: a first data set, a second data set, and a third data set, wherein the first data set uses 128-channel resting-state EEG signals of depression and normal subjects in the MODMA data set, the second data set uses 3-channel resting-state EEG signals of depression and normal subjects in the MODMA data set of the same subjects as in the first data set, and the third data set uses another 3-channel resting-state EEG signal data set of depression and normal subjects collected from more subjects; The data preprocessing module is used to perform denoising and feature extraction on resting-state EEG signal data; The knowledge distillation module is provided with a teacher model and a student model; the teacher model is trained using the first data set, and during the training process, the knowledge extracted from the 128-channel EEG signal is fed to the student model; The student model is inputted with the data of the second data set, and receives the knowledge fed by the teacher model to learn the common knowledge of the 128-channel EEG signals in the first data set and the 3-channel EEG signals in the second data set, as well as the unique knowledge of the 3-channel EEG signals in the second data set; The meta-transfer learning module uses the resting-state EEG signal data of the third data set to fine-tune the student model; The depression recognition module is used to implement a depression recognition task for the resting-state EEG signal data to be recognized based on the fine-tuned student model.

2. The EEG depression recognition system based on knowledge distillation and meta-transfer learning as claimed in claim 1, characterized in that: The student model is based on an improved EEGNet neural network structure; the teacher model is based on an improved GCN neural network structure.

3. The EEG depression recognition system based on knowledge distillation and meta-transfer learning as claimed in claim 2, characterized in that: The teacher model includes two series-connected graph convolution modules GCN and a fully connected layer and a softmax layer connected in sequence; The student model includes four two-dimensional convolutional layers connected in series, and a sub-classifier module consisting of a bottleneck layer, a fully connected layer, and a SoftMax layer connected in sequence is added after each two-dimensional convolutional layer; each of the sub-classifier modules outputs a set of probability prediction values; The teacher model and the student model are trained simultaneously, and the loss used in the training update process is as follows: The first loss is used to calculate the loss between the teacher model's predicted label and the actual data label; The second loss is used to calculate the loss between the labels predicted by each sub-classifier module of the student model and the actual data labels; The third loss is used to calculate the loss between the probability distribution predicted by the SoftMax layer of each sub-classifier module of the student model and the probability distribution predicted by the SoftMax layer of the teacher model; The fourth is used to calculate the loss between the data feature distribution after the fully connected layer of each sub-classifier module of the student model and the data feature distribution after the fully connected layer of the teacher model.

Citation Information

Cited By

  • ALS patient depression risk prediction model construction method and device

    CN119811666A

  • A method and device for constructing a prediction model of depression risk in ALS patients

    CN119811666B

  • Face image depression recognition system based on instance-level relation distillation

    CN120032415A

  • Depression diagnosis and treatment system capable of explaining electroencephalogram adaptive transfer learning

    CN122376103A

  • Three-lead electroencephalogram depression detection method based on knowledge distillation and time feature enhancement

    CN122515785A