Multi-parameter fusion mood disorder assessment system based on time sequence dynamic graph network

Abstract

The invention provides a multi-parameter fusion mood disorder assessment system based on a time sequence dynamic graph network, and the system comprises a data collection unit which is used for collecting an electroencephalogram signal of a to-be-assessed patient; the electroencephalogram coding unit is used for extracting electroencephalogram features corresponding to the electroencephalogram signals; the time sequence dynamic graph network feature extraction unit is used for constructing a time sequence dynamic graph network sequence related to various electroencephalogram parameter combinations and extracting related time sequence dynamic graph network features; the demographic information coding unit is used for coding the demographic information to obtain demographic coding features; and the hierarchical multi-level gating unit is used for predicting and obtaining a mood disorder evaluation result based on the sequential dynamic graph network characteristics corresponding to the electroencephalogram parameter combinations obtained by screening of the sequentially arranged multi-level gating layers. According to the method, the problem that in the prior art, a mood disorder evaluation system does not consider the correlation between multi-parameter combination and tasks and does not perform brain network model screening in a hierarchical gating mode, so that the evaluation accuracy is limited is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of clinical medicine, and particularly relates to a multi-parameter fusion mood disorder assessment system based on a temporal dynamic graph network. Background Art

[0002] With the development of society, the high incidence of depression and bipolar disorder has attracted wide social attention. Psychiatrists are facing increasingly heavy work in differentiating normal / depressed moods, unipolar depression / bipolar depression, and the differentiation of unipolar depression / bipolar depression is a difficult point in clinical differentiation work.

[0003] Electroencephalogram (EEG) is a commonly used measurement method for recording the electrical activities generated by the cerebral cortex. It has the characteristics of high resolution, non-invasiveness, and low cost, and can reflect the neural activities of various functional areas of the human brain. Therefore, it is applied in the diagnosis of brain diseases, motor recovery, and the assessment of nervous system diseases. In recent years, in the fields of psychology and cognitive neuroscience, research on exploring mood disorders based on EEG has also been on the rise. However, the current mood disorder assessment systems based on EEG usually start from the original EEG signals and are easily affected by EEG signal noise; and in the modeling process, there is a lack of dynamic modeling of the brain network and the decoupling of features in terms of frequency bands, brain regions, and observation time scales, resulting in poor interpretability and generalization of the assessment systems. Summary of the Invention

[0004] In view of the above analysis, the present invention aims to provide a multi-parameter fusion mood disorder assessment system based on a temporal dynamic graph network to solve the problem that the existing mood disorder assessment systems lack dynamic modeling of the brain network in the modeling process, resulting in poor interpretability and generalization of the assessment systems.

[0005] The object of the present invention is mainly achieved through the following technical solutions: The present invention provides a multi-parameter fusion mood disorder assessment system based on a temporal dynamic graph network, which system includes: A data acquisition unit for acquiring the EEG signals of the patient to be evaluated; An EEG encoding unit for encoding the EEG signals into latent representations to obtain the EEG features corresponding to the EEG signals; A temporal dynamic graph network feature extraction unit for constructing various combinations of EEG parameters, respectively establishing temporal dynamic graph network sequences of the EEG signals based on the EEG features corresponding to various combinations of EEG parameters, and extracting features from each temporal dynamic graph network sequence to obtain temporal dynamic graph network features related to each combination of EEG parameters; A demographic information encoding unit for encoding and combining the demographic information of the patient to be evaluated and the state information during EEG acquisition to obtain demographic encoding features; A hierarchical multi-level gating unit is used to screen through multiple sequentially set gating layers based on the EEG features and demographic coding features to obtain a task-related EEG parameter combination; and based on the temporal dynamic graph network features related to the screened EEG parameter combination, a mood disorder assessment result is predicted.

[0006] Further, the parameters in the EEG parameter combination include frequency band, brain region, and observation time unit.

[0007] Further, the EEG parameter combination is constructed by the following method: Pairwise combine different brain regions to obtain multiple brain region combinations; Set multiple observation time units with different lengths; Combine different frequency bands, brain region combinations, and observation time units respectively in a preset level order to obtain multiple EEG parameter combinations and sub-class parameter combinations corresponding to each parameter.

[0008] Further, the temporal dynamic graph network feature extraction unit obtains the temporal dynamic graph network features related to each parameter combination through a dynamic edge convolution network and a temporal embedding module; wherein, The dynamic edge convolution network is used to construct a dynamic graph network sequence related to various EEG parameter combinations; The temporal embedding module is used to learn the feature relationship of the dynamic graph network sequence through a long short-term memory network and perform feature extraction through a fully connected layer to obtain the temporal dynamic graph network features.

[0009] Further, the dynamic edge convolution network constructs the dynamic graph network sequence related to each EEG parameter combination by the following method: Slice the EEG signal into multiple observation windows according to the length of the observation time unit in the EEG parameter combination to be processed; For each observation window, take the channel dimension of the EEG features corresponding to the EEG parameter combination as the nodes of the graph; For each node, calculate its distance from other nodes through the Euclidean distance, sort the distances in ascending order, select n nodes with the closest distances as neighbors, and each neighbor node forms an edge with this node; Concatenate the features of the two nodes corresponding to each edge to obtain the feature of each edge, and transform the feature of each edge into the feature of the target quantity through a multi-layer perceptron; Aggregate the features of the target quantity of all edges corresponding to each node, and perform a global pooling operation on the aggregated features of all nodes to obtain the one-dimensional feature corresponding to this observation window; Concatenate the one-dimensional features corresponding to all observation windows in chronological order to obtain the dynamic graph network sequence related to the EEG parameter combination.

[0010] Furthermore, the long short-term memory network is formed by connecting multiple sequential network units in series; The sequential embedding module is used to learn the feature relationships of the dynamic graph network sequence through the long short-term memory network and perform feature extraction through a fully connected layer, including: Align each observation window in the EEG signal with each sequential network unit of the long short-term memory network; Input the one-dimensional features corresponding to each observation window into the corresponding sequential network unit; flatten the outputs of multiple sequential network units into one dimension and input them into the fully connected layer to extract the sequential dynamic graph network features.

[0011] Furthermore, the hierarchical multi-level gating unit includes a frequency band gating layer, a brain region combination gating layer, an observation time unit gating layer, and a fusion evaluation layer; The frequency band gating layer is used to screen out a preset number of task-related frequency bands based on the EEG features corresponding to different frequency bands; and perform classification prediction on the sequential dynamic graph network features related to each subclass parameter combination corresponding to the task-related frequency bands to obtain the frequency band-related evaluation result; The brain region combination gating layer uses the task-related frequency bands screened out as a constraint to screen out the task-related brain region combinations, and performs classification prediction on the sequential dynamic graph network features related to the subclass parameter combinations corresponding to the combination of the task-related brain region combinations and the frequency bands to obtain the brain region-related evaluation result; The observation time unit gating layer uses the task-related frequency bands and brain region combinations screened out as constraints to screen out the task-related observation time units, and performs classification prediction on the sequential dynamic graph network features related to the parameter combinations corresponding to the task-related frequency bands, brain region combinations, and observation time units to obtain the observation time-related evaluation result; The fusion evaluation layer is used to perform weighted fusion on the frequency band-related evaluation result, the brain region-related evaluation result, and the observation time-related evaluation result to obtain the final mood disorder evaluation result.

[0012] Furthermore, the frequency band gating layer includes an input layer, a frequency band sub-class dense gating layer, a frequency band sparse gating layer, and a frequency band classification prediction layer; The input layer is used to concatenate the EEG features corresponding to each frequency band with the demographic coding features and input them into the frequency band sub-class dense gating layer and the frequency band sparse gating layer respectively; The output quantity of the frequency sub - level dense gating layer is the same as the number of subclass parameter combinations corresponding to each frequency band, which is used to extract features from the input data, and obtain the weights corresponding to each subclass parameter combination through an activation function; based on the weights, the temporal dynamic graph network features corresponding to each subclass parameter combination of each frequency band are weighted and fused to obtain the fusion features corresponding to each frequency band; The output quantity of the frequency band sparse gating layer is the same as the number of frequency bands, which is used to extract features from the input data, and obtain the task relevance indicators of each frequency band through an activation function, arrange the task relevance indicators in descending order, and screen to obtain a preset number of task - related frequency bands; The frequency band classification prediction layer is used to receive the fusion features corresponding to the task - related frequency bands and predict the frequency band - related classification results.

[0013] Further, the brain region combination gating layer includes an input layer, a brain region sub - level dense gating layer, a brain region sparse gating layer, and a brain region classification prediction layer; The input layer is used to flatten the channel dimension and feature dimension of the task - related frequency bands corresponding to each brain region combination, splice them with the demographic coding features, and respectively input them into the brain region sub - level dense gating layer and the brain region sparse gating layer; The output quantity of the brain region sub - level dense gating layer is the number of alternative parameter combinations composed of the task - related frequency bands corresponding to each brain region combination and each observation time unit; the brain region sub - level dense gating layer is used to extract features from the received data, and obtain the weights corresponding to each alternative parameter combination of each brain region combination through an activation function; and based on the weights, the temporal dynamic graph network features corresponding to each alternative parameter combination of each brain region combination are weighted and fused to obtain the fusion features corresponding to each brain region combination; The output layer quantity of the brain region sparse gating layer is the number of brain region combinations, which is used to extract features from the received data, and obtain the task relevance indicators of each brain region combination through an activation function, arrange the task relevance indicators in descending order to obtain a preset number of task - related brain region combinations; The brain region classification prediction layer is used to receive the fusion features corresponding to the task - related brain region combinations and obtain the brain region - related classification results.

[0014] Further, the observation time unit gating layer includes an input layer, an observation time sparse gating layer, and an observation time unit classification prediction layer; The input layer is used to obtain the alternative EEG features related to the length of each observation time unit based on the task - related brain region combinations and the task - related frequency bands, flatten the alternative EEG features related to the length of each observation time unit, splice them with the demographic coding features respectively, and input them into the observation time gating layer; The number of output nodes of the observed time sparse gating layer is the number of observed time unit length categories, which is used to extract features from the received data, and obtain the task relevance indicators for each observed time unit length through an activation function, and select the observed time unit length with the largest task relevance indicator as the task-related observed time unit; The observed time unit classification prediction layer is used to receive the temporal dynamic graph network features corresponding to the task-related observed time units and the alternative subclass parameter combinations, and obtain the observed time unit-related classification results.

[0015] Beneficial effects of the present technical solution: 1. The present invention proposes an electroencephalogram signal analysis technology based on a temporal dynamic graph network for electroencephalogram feature modeling with multiple frequency bands, multiple brain regions, and multiple observed time units; and encodes the demographic information of the subjects into a hierarchical gating model, screens task-related parameters layer by layer in the dimensions of frequency band, brain region, and observed time unit, and performs weighted fusion on the selected parameter features for mood disorder assessment, improving the accuracy and efficiency of mood disorder assessment, especially suitable for the diagnosis of cross-subject mood disorders, and the assessment system has good generalization and assessment accuracy; 2. The present invention selects brain regions related to mood disorders, combines the brain regions in pairs, and combines the frequency band and the length of the observed time unit to construct evaluation data, and can accurately extract the features related to mood disorders based on the latent features of electroencephalogram signals, using the linkage between brain regions and between the frequency band and the observed time for mood disorder assessment, greatly improving the accuracy of mood disorder assessment.

[0016] Other features and advantages of the present invention will be described in the following specification, and some of them will become obvious from the specification, or be understood by implementing the present invention. The objectives and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the written specification, claims, and drawings. Brief Description of the Drawings

[0017] The drawings are only for the purpose of showing specific embodiments, and are not considered to be a limitation of the present invention. Throughout the drawings, the same reference numerals represent the same components; Figure 1 It is a schematic diagram of a multi-parameter fusion mood disorder assessment system based on a temporal dynamic graph network according to an embodiment of the present invention; Figure 2 It is a schematic diagram of the function of an electroencephalogram encoding model according to an embodiment of the present invention; Figure 3 It is a schematic diagram of constructing temporal dynamic graph network features based on electroencephalogram multi-parameter combinations according to an embodiment of the present invention; Figure 4 It is a model training flowchart according to an embodiment of the present invention. Specific Embodiment

[0018] The preferred embodiments of the present invention will be specifically described below with reference to the accompanying drawings. The accompanying drawings form a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0019] An embodiment of the present invention provides a multi-parameter fusion mood disorder assessment system based on a temporal dynamic graph network, as Figure 1 shown. The system includes: A data acquisition unit for acquiring the electroencephalogram (EEG) signals of the patient to be evaluated; An EEG encoding unit for encoding the EEG signals into latent representations to obtain the EEG features corresponding to the EEG signals; A temporal dynamic graph network feature extraction unit for constructing various combinations of EEG parameters, respectively establishing the temporal dynamic graph network sequences of the EEG signals based on the EEG features corresponding to various combinations of EEG parameters, and extracting features from each temporal dynamic graph network sequence to obtain the temporal dynamic graph network features related to each parameter combination; A demographic information encoding unit for encoding and combining the demographic information of the patient to be evaluated and the state information during EEG acquisition to obtain demographic encoding features; A hierarchical multi-level gating unit for respectively screening based on the EEG features and the demographic encoding features through sequentially arranged multi-level gating layers to obtain the task-related combinations of EEG parameters; and predicting the mood disorder assessment result based on the temporal dynamic graph network features corresponding to the screened combinations of EEG parameters.

[0020] Specifically, the data acquisition unit in this embodiment uses a resting-state multi-channel scalp EEG acquisition device to acquire the electrophysiological signals of the subjects for subsequent EEG signal analysis and mood disorder assessment.

[0021] After the EEG signals are acquired, the EEG signals are first preprocessed by a data preprocessing unit, which specifically includes downsampling, re-referencing, notch filtering to remove power frequency, band-pass filtering, using the ICA method to remove electrooculogram, electrocardiogram, and electromyogram, removing abnormal amplitude segments, and segmenting the processed segments into EEG segments of a fixed time length; where removing abnormal amplitude segments includes: first performing time slicing on the band-pass filtered EEG data, then calculating the amplitude within the time slice, and deleting the time slices whose amplitudes are not within the preset range. In this embodiment, the time length of the EEG segments is set to 120 seconds, and the average value of the assessment results of all EEG segments is taken as the final assessment result.

[0022] After preprocessing the EEG signals, the EEG encoding unit encodes each EEG segment of the EEG signals into latent representations and performs brain region grouping: Specifically, as Figure 2 shown, through the pre-trained EEG encoding model, each 1-second single-channel EEG signal is encoded into a latent representation. For different frequency bands of EEG, there are corresponding feature domains in the latent representation. According to the electrode positions and brain regions of the EEG signals, the EEG signals corresponding to different channels are grouped.

[0023] In this embodiment, the existing EEG encoding model VAEEG is adopted to encode the 1-second single-channel EEG signal into the latent representation space. Each preprocessed signal will be converted into feature signals under different frequency band representations such as delta[1,4]Hz, theta[4-8Hz], alpha[8-13]Hz, beta1[13-25]Hz, beta2[25-40]Hz, etc. The dimensionality of the representation space corresponding to different frequency bands of each 1-second segment is delta_zim, theta_zim, alpha_zim, beta1_zim, beta2_zim. For each different acquired electrode position, according to the brain functional regions divided into frontal region, parietal region, central region, left temporal region, right temporal region, occipital region, the latent representation corresponding to each brain region is divided into the corresponding region.

[0024] In this embodiment, through the EEG encoding unit, the complex EEG signals are stably feature-extracted by reducing the dimensionality of the EEG signals, achieving a certain denoising effect.

[0025] Among them, for the next step of dividing the observation time units, the EEG segments for feature extraction in this step should be as short as possible, or directly use the observation time units for feature extraction.

[0026] Further, the temporal dynamic graph network feature extraction unit constructs the EEG parameter combinations by the following method: Combining different brain regions pairwise to obtain multiple brain region combinations; Setting multiple observation time units with different lengths; Combining different frequency bands, brain region combinations, and observation time units in sequence according to the preset level order to obtain multiple EEG parameter combinations and the subclass parameter combinations corresponding to each parameter, that is, the sum of the subclass parameter combinations corresponding to each parameter constitutes all the EEG parameter combinations.

[0027] Specifically, for the frequency bands, in this embodiment, five frequency bands with relatively high relevance to mood disorders, namely delta, theta, alpha, beta1, and beta2, are selected; for the brain regions, the types selected are the frontal region, parietal region, central region, left temporal region, right temporal region, and occipital region; for the length of the observation time unit, three lengths, short, medium, and long, are preset, corresponding to the observation time unit lengths of 5 s, 10 s, and 15 s respectively; in this embodiment, the frequency band category is used as the first-level parameter, the brain region category corresponds to the second-level parameter, and the observation time category corresponds to the third-level parameter; fixing the level order and combining different parameters, the obtained electroencephalogram (EEG) parameter combinations are, for example: ; Among them, the number of obtained EEG parameter combinations is: ; For the connection relationships such as frontal region - parietal region and frontal region - central region in the brain region combinations, the number of obtained brain region combinations is: ; For each level of parameter, there is a corresponding sub - parameter combination. Exemplarily, for the delta frequency band, the number of its corresponding sub - parameter combinations is: ; Such as: .

[0028] Furthermore, the temporal dynamic graph network feature extraction unit obtains the temporal dynamic graph network features related to each parameter combination through a dynamic edge convolution network and a temporal embedding module; The dynamic edge convolution network is used to construct a dynamic graph network sequence related to various EEG parameter combinations; The temporal embedding module is used to learn the feature relationships of the dynamic graph network sequence through a long short - term memory network and perform feature extraction through a fully - connected layer to obtain the temporal dynamic graph network features.

[0029] Specifically, the dynamic edge convolution network constructs the dynamic graph network sequence related to each EEG parameter combination through the following method based on the feature similarity: The EEG signals are sliced into multiple observation windows according to the length of the observation time unit in the EEG parameter combinations to be processed; For each observation window, the channel dimension of the EEG features corresponding to the EEG parameter combination is used as the nodes of the graph; For each node, its distance from other nodes is calculated through the Euclidean distance, the distances are arranged in ascending order, and n nodes with the closest distances are selected as neighbors, and each neighbor node forms an edge with this node; The features of the two nodes corresponding to each edge are concatenated to obtain the feature of each edge, and the feature of each edge is transformed into the feature of the target quantity through a multi - layer perceptron; Aggregate the features of the target quantities of all the edges corresponding to each node, and perform a global pooling operation on the aggregated features of all the nodes to obtain a one-dimensional feature corresponding to the observation window. Concatenate the one-dimensional features corresponding to all the observation windows in chronological order to obtain the dynamic graph network sequence related to the EEG parameter combination.

[0030] This embodiment is different from the static brain network analysis based on anatomical connection and functional connection. This embodiment adopts a completely data-driven method to dynamically construct a graph structure according to the similarity of features in each time window. The construction method of the K-nearest neighbor graph can be used to construct the dynamic graph of each observation window. For each observation window, the input dimension of its corresponding EEG feature is (number of channels, number of features), where the number of channels is all the channels included in the corresponding brain region combination. Take the channel dimension of the EEG feature as the nodes of the graph. For each node, calculate its distance from other nodes through the Euclidean distance, sort the distances in ascending order, and select the top 4 nearest nodes as neighbors. Each node forms an edge with this node to construct the graph structure of each observation window.

[0031] Then perform feature aggregation and transformation on the graph structure, including taking the features of the two nodes splicing each edge as new features, and connecting the nonlinear multi-layer perceptron to transform the new features into the features of the target quantity; in order to reduce the complexity of the model, the number of layers of the multi-layer perceptron is 1 layer; then perform message aggregation on the features of the target quantity corresponding to each edge, that is, for each node, aggregate the features of the target quantity corresponding to all the edges. The aggregation method adopted by the present invention is mean aggregation.

[0032] After performing message aggregation on each node, perform average pooling on the features of all the nodes of each observation window, so that the graph structure features of each observation window are compressed into a one-dimensional feature, denoted as EdgeConv_zim; for a fixed frequency band and brain region combination, only need to focus on the internal dynamic changes between a specific frequency band and a specific brain region. Corresponding to each observation window, a graph structure will be obtained. This graph structure is a dynamic graph that adaptively aggregates edge nodes. Concatenate the graph structures of multiple observation windows, that is, form a dynamic graph network sequence.

[0033] Preferably, the long short-term memory network is formed by connecting multiple time series network units in series; The time series embedding module is used to learn the feature relationship of the dynamic graph network sequence through the long short-term memory network, and perform feature extraction through a fully connected layer, as Figure 3 shown, including: Align each observation window in the EEG signal with each time series network unit of the long short-term memory network; Input the one-dimensional features corresponding to each observation window into the corresponding time series network unit; flatten the outputs of multiple time series network units into one dimension and input them into a fully connected layer to extract the time series dynamic graph network features.

[0034] The present invention uses a long short-term memory network to learn the relationship of dynamic graph network features. This network is composed of time series network units (lstm cells) in series with the number of observation windows. The time series information transmission direction is single-phase. The hidden layer of each time series network unit is 1 layer, and the output dimension is set to LSTM_zim; align the features of each observation window with the time series network unit (LSTMcell), and use the output EdgeConv_zim of the dynamic edge convolution network as the input of the corresponding time series embedding module; then flatten the outputs of multiple time series network units into one dimension to obtain features with a dimension of Length; connect the obtained features to a fully connected layer with the target output of EdgeLstm_zim to extract the time series dynamic graph network features.

[0035] The time series dynamic graph network feature extraction model is obtained by iteratively training through a pre-constructed training sample set; When constructing the training sample set, if it is applied to depression / non-depression assessment, obtain the electroencephalogram features corresponding to multiple mood disorder patients and healthy subjects, and label them with normal or mood disorder labels; if it is applied to unipolar / bipolar depression assessment, obtain the electroencephalogram features corresponding to multiple unipolar depression and bipolar depression mood disorder patients, and label them with unipolar depression or bipolar depression labels to construct the training sample set; When training the time series dynamic graph feature extraction model, group each electroencephalogram feature in the training sample set according to each combination of electroencephalogram parameters to obtain the training set corresponding to each combination of electroencephalogram parameters; connect the time series dynamic graph network feature extraction model to a classifier composed of 2 layers of multi-layer perceptrons, and use each training set to optimize the model parameters with cross-entropy as the loss function. Use the validation set to evaluate its discrimination effect. When the sensitivity and specificity of the validation set are not lower than a certain threshold, save the pre-trained model. Otherwise, adjust the parameters of the model within a preset number of times until the validation set meets the standard. If it exceeds the preset limit, ignore the model to participate in the establishment of the subsequent evaluation model, and save the model parameters obtained by training each training set to obtain the dynamic graph network feature extraction model related to each combination of electroencephalogram parameters.

[0036] The demographic information encoding unit obtains the demographic encoding features through the following method: Read the gender, age of the subject, and the open / closed eye information when collecting the electroencephalogram signal, and perform preprocessing respectively through the following method: For age information, use standardization to unify the numerical range among different features and avoid large differences in gradients during the model learning process; for example, use standardization (x - x_mean), x - std, or normalization methods. For gender information, use one - hot encoding, encoding male as (0, 1) and female as (1, 0). For eyes - open / closed information, use one - hot encoding, encoding eyes - open as (0, 1) and eyes - closed as (1, 0). For each dimension of the pre - processed information, input it into the corresponding information encoding module composed of multiple fully - connected layers respectively, and perform non - linear transformation using an activation function. Encode and merge the information obtained after non - linear transformation, and map it to a unified representation space to obtain demographic encoding features.

[0037] Furthermore, the hierarchical multi - level gating unit includes a frequency - band gating layer, a brain - region combination gating layer, an observation - time unit gating layer, and a fusion evaluation layer. The frequency - band gating layer is used to screen a preset number of task - related frequency bands based on the EEG features corresponding to different frequency bands; and perform classification prediction based on the temporal dynamic graph network features related to each subclass parameter combination corresponding to the task - related frequency bands to obtain frequency - band - related evaluation results. The brain - region combination gating layer uses the screened task - related frequency bands as a constraint to screen task - related brain - region combinations, and performs classification prediction based on the temporal dynamic graph network features related to the subclass parameter combination corresponding to the combination of the task - related brain - region combinations and frequency bands to obtain brain - region - related evaluation results. The observation - time unit gating layer uses the screened task - related frequency bands and brain - region combinations as constraints to screen task - related observation - time units, and performs classification prediction based on the temporal dynamic graph network features related to the parameter combination corresponding to the task - related frequency bands, brain - region combinations, and observation - time units to obtain observation - time - related evaluation results. The fusion evaluation layer is used to perform weighted fusion on the frequency - band - related evaluation results, brain - region - related evaluation results, and observation - time - related evaluation results to obtain the final mood disorder evaluation result.

[0038] This embodiment is divided into multiple gating layers based on each parameter angle. The first level is the frequency - band gating layer, the second level is the brain - region gating layer, and the third level is the observation - time gating layer. For the first - level gating layer, it is further divided into sub - levels of multi - frequency - band features, from the delta sub - gating layer to the beta2 sub - gating layer. For the second - level brain - region gating layer, brain - network combinations are composed of any two brain networks, from the frontal - parietal sub - gating layer to the left - temporal - occipital sub - gating layer. For the third - level observation - time gating layer, no sub - gating - layer division is required.

[0039] The band gating layer includes an input layer, a band sub-level dense gating layer, a band sparse gating layer and a band classification prediction layer; The input layer is used to concatenate the EEG features corresponding to each frequency band with the demographic coding features, and input them into the frequency band sub-level dense gating layer and the frequency band sparse gating layer respectively; The frequency band sub-level dense gating layer and the frequency band sparse gating layer are both constructed based on a multi-layer perceptron; The number of outputs of the frequency band sub-level dense gating layer is the same as the number of sub-class parameter combinations corresponding to each frequency band, and is used to extract features from the input data, and obtain weights corresponding to each sub-class parameter combination through an activation function; based on the weights, the time series dynamic graph network features corresponding to each sub-class parameter combination corresponding to each frequency band are weightedly fused to obtain fused features corresponding to each frequency band; The number of outputs of the frequency band sparse gating layer is the same as the number of frequency bands, and is used to extract features from the input data, and obtain the task relevance index of each frequency band through an activation function, and arrange the task relevance indexes in descending order to screen and obtain a preset number of task-related frequency bands; The frequency band classification prediction layer is used to receive fused features corresponding to task-related frequency bands and predict frequency band-related classification results.

[0040] Specifically, for the frequency band sub-level gating layer, since the pre-selected frequency band is fixed, the EEG features associated with the frequency band dimension are spliced with the demographic coding features, and then connected to the multi-layer perceptrons with the same number of outputs as the target hyperparameter combinations. After the output features pass through the softmax layer, the weights of the features corresponding to each parameter combination are obtained. Based on the obtained weights, the corresponding time series dynamic graph network features are weighted fused to obtain the fused features corresponding to each frequency band, expressed as EdgeLstm_delta, EdgeLstm_theta, EdgeLstm_alpha, EdgeLstm_beta1, EdgeLstm_beta2. In order to ensure the diversity of the features in the early stage of encoding of the model and to enhance stability, dense gating is used to select the features corresponding to the sub-level parameters for fusion.

[0041] For the frequency band gating layer, the multilayer perceptron uses the number of frequency bands as the output dimension, selects the output features and sorts them into the top 2 frequency bands after passing through the softmax layer, and this embodiment adopts sparse gating to select the target frequency band, with the aim of selecting the frequency band most relevant to the task purpose.

[0042] The brain region combination gating layer includes an input layer, a brain region sub-level dense gating layer, a brain region sparse gating layer and a brain region classification prediction layer; The input layer is used to flatten the channel dimension and feature dimension of the task-related frequency bands corresponding to each brain region combination, concatenate them with the demographic coding features, and respectively input them into the sub-brain-region dense gating layer and the brain-region sparse gating layer; Both the sub-brain-region dense gating layer and the brain-region sparse gating layer are trained based on multi-layer perceptrons; The output quantity of the sub-brain-region dense gating layer is the quantity of alternative parameter combinations composed of the task-related frequency bands corresponding to each brain region combination and each observation time unit; the sub-brain-region dense gating layer is used to extract features from the received data, and obtain the weights of each alternative parameter combination corresponding to each brain region combination through an activation function; and based on the weights, perform weighted fusion on the temporal dynamic graph network features of each alternative parameter combination corresponding to each brain region combination to obtain the fusion features corresponding to each brain region combination; The output layer quantity of the brain-region sparse gating layer is the quantity of brain region combinations, which is used to extract features from the received data, and obtain the task relevance indexes of each brain region combination through an activation function, sort the task relevance indexes in descending order to obtain a preset quantity of task-related brain region combinations; The brain-region classification prediction layer is used to receive the fusion features corresponding to the task-related brain region combinations to obtain the brain-region related classification results.

[0043] The observation time unit gating layer includes an input layer, an observation time sparse gating layer, and an observation time unit classification prediction layer; The input layer is used to obtain alternative EEG features related to the length of each observation time unit based on the task-related brain region combinations and the task-related frequency bands, flatten the alternative EEG features related to the length of each observation time unit, concatenate them with the demographic coding features respectively, and input them into the observation time gating layer; The output node quantity of the observation time sparse gating layer is the quantity of observation time unit length categories, which is used to extract features from the received data, and obtain the task relevance indexes of the lengths of each observation time unit through an activation function, and select the observation time unit length with the largest task relevance index as the task-related observation time unit; The observation time unit classification prediction layer is used to receive the temporal dynamic graph network features corresponding to the task-related observation time unit and the alternative subclass parameter combinations to obtain the observation time unit related classification results.

[0044] Specifically, for the sub-level gating layer of brain region combinations, the alternative models of the gating modules in this layer are constrained by the selected frequency bands in the frequency band gating layer. Since the preselected brain region selection is fixed, electrodes corresponding to the selected brain regions need to be selected in the channel dimension of each frequency band. The channel part and the feature part of the EEG features are flattened, concatenated with the demographic coding features, and input to a multi-layer perceptron connected to the number of alternative models in this layer as the output. Then, the output is passed through a softmax layer to obtain the weights of each alternative model. According to this weight, the corresponding temporal dynamic graph network features are weighted and fused to obtain the fused features corresponding to each brain region combination. To enhance the stability of the model and the diversity of features, in this embodiment, the features corresponding to the sub-level parameters are fused in a dense gating manner.

[0045] For the brain region combination gating layer, the multi-layer perceptron has the number of brain regions as the output dimension, and the selected output features are sorted as the top 2 corresponding frequency bands after passing through the softmax layer; in this embodiment, a sparse gating method is used to select the target brain regions, aiming to selectively extract the temporal dynamic graph network features related to the task objectives.

[0046] For the observation time unit gating layer, the parameters of this layer are constrained by the frequency bands and brain region combinations related to the task, and do not involve the sub-level gating layer. The alternative EEG features related to the length of each observation time unit are flattened and concatenated with the demographic coding features respectively, and then connected to a multi-layer perceptron with the number of parameters of the observation time unit as the number of output nodes. The selected output features are sorted as the top 1 corresponding observation time unit length after passing through the softmax layer. Here, a sparse gating method is used to select the observation time unit length, aiming to selectively select the observation time units related to the task.

[0047] The hierarchical multi-level gating unit is trained using the training sample set constructed during the training of the temporal dynamic graph network feature extraction model, and the temporal dynamic graph network features of each sample data related to each parameter combination obtained during the training of the temporal dynamic graph network feature extraction model; The specific training process is as Figure 4 shown. When the hierarchical multi-level gating unit is trained, it also includes hierarchically deleting redundant data from the temporal dynamic graph network features related to each parameter combination obtained by the temporal dynamic graph network feature extraction unit to obtain an alternative set of temporal dynamic graph network features for iterative optimization of the model, including: Dividing the data in the training sample set into K folds, and iteratively using the training data in the K folds to obtain the temporal dynamic graph network features related to all EEG parameter combinations; Fixing the number of samples, calculating the correlation between each parameter combination based on the temporal dynamic graph network features; and calculating the mean value of the correlation in the sample dimension, and marking the parameter combinations whose absolute value of the correlation is greater than the first preset threshold; After K iterations, remove the parameter combinations with the number of marked ones greater than the second preset threshold to obtain an alternative set of temporal dynamic graph network features.

[0048] For the subclass parameter combinations at each level, the features encoded by different temporal dynamic graph networks may have a high degree of correlation. In this step of the present invention, a method for reducing the redundancy of the model is introduced so that the subsequent gating network can achieve better weight allocation.

[0049] Specifically, based on the training set, divide the training set into K folds, iteratively use the training data in the K folds to obtain the temporal dynamic graph network features related to all parameter combinations. The feature dimension of each batch is (number of samples, number of hyperparameter combinations, EdgeLstm_zim). Fix the number of samples, and based on the features, calculate the correlation between each parameter combination, calculate the mean value of the correlation in the sample dimension to obtain a feature dimension of (number of parameter combinations, number of parameter combinations), and mark the dimensions with the absolute value of the correlation greater than 0.65. After iterating the above steps K times, remove the dimensions with the number of marked ones greater than 0.8*K. Finally, obtain an alternative set of temporal dynamic graph network models.

[0050] During the training process, in this embodiment, the dynamic graph network features corresponding to the frequency band gating layer, the brain network combination gating layer, and the observation time unit gating layer are respectively connected to a multi-layer perceptron to output a classification result, and the cross entropy is calculated separately for the results output by each level. Different weights are set for each cross entropy, and the total loss function of the model is defined as: ; ; ; ; When performing weighted fusion, the weights of each gating layer satisfy the following constraint conditions: ; Among them, is the total loss of the model, is the weight of the frequency band gating layer, is the weight of the brain region gating layer, is the weight of the time observation unit gating layer, is the cross-entropy loss of the frequency band gating layer, is the cross-entropy loss of the brain region gating layer, is the cross-entropy loss of the time observation unit gating layer, is the labeled label, is the predicted value of the frequency band gating layer, is the predicted value of the brain region gating layer, is the predicted value of the observation time unit gating layer; Since the model design progresses from the frequency band layer to the brain network combination layer to the observation time unit layer to design the gating mechanism and achieve feature selection of the dynamic graph network, the principle is that through the progressive feature angles and continuously refined feature backgrounds at each layer, better mood disorder assessment can be achieved. Therefore, when setting the cross-entropy weights, the weights of each layer should be maintained as: the weight of the observation time unit layer > the weight of the brain network combination layer > the weight of the frequency band layer.

[0051] In summary, a multi-parameter fusion mood disorder assessment system based on a time-series dynamic graph network provided by an embodiment of the present invention uses a hierarchical multi-level gating mechanism to implement an expert model screening method for adaptive subject demographic information, a method for gradually progressive modeling of the gating network based on multi-angle parameters (including frequency band, brain region combination, observation time unit length), establishes corresponding sub-level gates for each parameter category (each layer), uses a dense modeling method to increase the diversity of sub-level features in each sub-layer gate of each layer, and uses a sparse modeling method in the deep gate to increase the pertinence of the task discrimination effect; the present invention uses a two-stage training method for the time-series dynamic graph convolutional network and the gating network. When pre-training the spatio-temporal dynamic graph network, the demographic information of the subject is not added, and only the EEG representation is learned. When establishing the gating network, the demographic information is added to assist in the screening of multi-level alternative graph network features. Moreover, the present invention uses a time-series dynamic edge convolutional neural network for modeling to achieve feature extraction, has a stronger ability to extract complex brain function features, and using the system of the present invention for mood disorder assessment greatly improves the accuracy of mood disorder assessment.

[0052] Those skilled in the art can understand that all or part of the processes for implementing the above embodiment methods can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium is a disk, an optical disc, a read-only memory, or a random access memory, etc.

[0053] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A multi-parameter fusion mood disorder assessment system based on a temporal dynamic graph network, characterized in that Comprising: A data acquisition unit for acquiring the electroencephalogram (EEG) signals of a patient to be evaluated; An EEG encoding unit for encoding the EEG signals into a latent representation to obtain the EEG features corresponding to the EEG signals; A temporal dynamic graph network feature extraction unit for constructing multiple combinations of EEG parameters, respectively establishing the temporal dynamic graph network sequences of the EEG signals based on the EEG features corresponding to various combinations of EEG parameters, and extracting features from each temporal dynamic graph network sequence to obtain the temporal dynamic graph network features related to each combination of EEG parameters; A demographic information encoding unit for encoding and combining the demographic information of the patient to be evaluated and the state information during EEG acquisition to obtain demographic encoding features; A hierarchical multi-level gating unit for respectively performing screening through multiple sequentially set gating layers based on the EEG features and the demographic encoding features to obtain the task-related combination of EEG parameters; and predicting the mood disorder assessment result based on the temporal dynamic graph network features related to the screened combination of EEG parameters.

2. The multi-parameter fusion mood disorder assessment system based on the temporal dynamic graph network according to claim 1, wherein The parameters in the combination of EEG parameters include frequency bands, brain regions, and observation time units.

3. The multi-parameter fusion mood disorder assessment system based on the temporal dynamic graph network according to claim 2, wherein The combination of EEG parameters is constructed by the following method: Pairwise combining different brain regions to obtain multiple combinations of brain regions; Setting multiple observation time units with different lengths; Combining different frequency bands, combinations of brain regions, and observation time units respectively in a preset level order to obtain multiple combinations of EEG parameters and the sub-parameter combinations corresponding to each parameter.

4. The multi-parameter fusion mood disorder assessment system based on the temporal dynamic graph network according to claim 1, wherein The temporal dynamic graph network feature extraction unit obtains the temporal dynamic graph network features related to each parameter combination through a dynamic edge convolution network and a temporal embedding module; wherein, The dynamic edge convolution network is used to construct the dynamic graph network sequences related to various combinations of EEG parameters; The temporal embedding module is used to learn the feature relationships of the dynamic graph network sequences through a long short-term memory network and perform feature extraction through a fully connected layer to obtain the temporal dynamic graph network features.

5. The multi-parameter fusion mood disorder assessment system based on the temporal dynamic graph network according to claim 4, characterized in that The dynamic edge convolution network constructs the dynamic graph network sequences related to each combination of EEG parameters based on the similarity of features through the following method: Segmenting the EEG signals into multiple observation windows according to the length of the observation time unit in the combination of EEG parameters to be processed; For each observation window, taking the channel dimension of the EEG features corresponding to the combination of EEG parameters as the nodes of the graph; For each node, calculating its distance from other nodes through the Euclidean distance, arranging the distances in ascending order, and selecting n nodes with the closest distances as neighbors, and each neighbor node forms an edge with this node; Concatenating the features of the two nodes corresponding to each edge to obtain the feature of each edge, and converting the feature of each edge into the feature of the target quantity through a multi-layer perceptron; Aggregating the features of the target quantity of all the edges corresponding to each node, and performing a global pooling operation on the aggregated features of all the nodes to obtain the one-dimensional feature corresponding to this observation window; Concatenating the one-dimensional features corresponding to all the observation windows in chronological order to obtain the dynamic graph network sequence related to the combination of EEG parameters.

6. The multi-parameter fusion mood disorder assessment system based on the temporal dynamic graph network according to claim 5, wherein The long short-term memory network is composed of multiple sequentially connected temporal network units; The timing embedding module is used to learn the feature relationships of the dynamic graph network sequence through a long short-term memory network and perform feature extraction through a fully connected layer, including: Aligning each observation window in the EEG signal with each timing network unit of the long short-term memory network; Inputting the one-dimensional features corresponding to each observation window into the corresponding timing network unit; flattening the outputs of multiple timing network units into one dimension and inputting them into the fully connected layer to extract the timing dynamic graph network features.

7. The multi-parameter fusion mood disorder assessment system based on the temporal dynamic graph network according to claim 6, characterized in that, The hierarchical multi-level gating unit includes a frequency band gating layer, a brain region combination gating layer, an observation time unit gating layer, and a fusion evaluation layer; The frequency band gating layer is used to screen out a preset number of task-related frequency bands based on the EEG features corresponding to different frequency bands; and classify and predict the timing dynamic graph network features related to the parameter combinations of each subclass corresponding to the task-related frequency bands to obtain the frequency band-related evaluation results; The brain region combination gating layer uses the task-related frequency bands screened out as a constraint to screen out the task-related brain region combinations, and classifies and predicts the timing dynamic graph network features related to the parameter combinations corresponding to the combination of the task-related brain region combinations and the frequency bands to obtain the brain region-related evaluation results; The observation time unit gating layer uses the task-related frequency bands and brain region combinations screened out as constraints to screen out the task-related observation time units, and classifies and predicts the timing dynamic graph network features related to the parameter combinations corresponding to the task-related frequency bands, brain region combinations, and observation time units to obtain the observation time-related evaluation results; The fusion evaluation layer is used to perform weighted fusion on the frequency band-related evaluation results, brain region-related evaluation results, and observation time-related evaluation results to obtain the final mood disorder evaluation results.

8. The multi-parameter fusion mood disorder assessment system based on the temporal dynamic graph network according to claim 7, characterized in that, The frequency band gating layer includes an input layer, a frequency band sub-dense gating layer, a frequency band sparse gating layer, and a frequency band classification prediction layer; The input layer is used to splice the EEG features corresponding to each frequency band with the demographic coding features and input them into the frequency band sub-dense gating layer and the frequency band sparse gating layer respectively; The output quantity of the frequency band sub-dense gating layer is the same as the number of parameter combinations of each subclass corresponding to each frequency band, and is used to extract features from the input data and obtain the weights corresponding to each subclass parameter combination through an activation function; Based on the weights, perform weighted fusion on the timing dynamic graph network features corresponding to the parameter combinations of each subclass corresponding to each frequency band to obtain the fusion features corresponding to each frequency band; The output quantity of the frequency band sparse gating layer is the same as the number of frequency bands, and is used to extract features from the input data and obtain the task relevance indicators of each frequency band through an activation function, sort the task relevance indicators in descending order, and screen out a preset number of task-related frequency bands; The frequency band classification prediction layer is used to receive the fusion features corresponding to the task-related frequency bands and predict the frequency band-related classification results.

9. The multi-parameter fusion mood disorder assessment system based on the temporal dynamic graph network according to claim 8, characterized in that, The brain region combination gating layer includes an input layer, a brain region sub-dense gating layer, a brain region sparse gating layer, and a brain region classification prediction layer; The input layer is used to flatten the channel dimension and feature dimension of the task-related frequency bands corresponding to each brain region combination, and splice them with the demographic coding features, and then input them into the brain region sub-dense gating layer and the brain region sparse gating layer respectively; The output quantity of the brain region sub-dense gating layer is the quantity of the alternative parameter combinations composed of the task-related frequency bands corresponding to each brain region combination and each observation time unit; the brain region sub-dense gating layer is used to extract features from the received data, and obtain the weights of each alternative parameter combination corresponding to each brain region combination through an activation function; and based on the weights, perform weighted fusion on the temporal dynamic graph network features of each alternative parameter combination corresponding to each brain region combination to obtain the fusion features corresponding to each brain region combination; The output layer quantity of the brain region sparse gating layer is the quantity of brain region combinations, which is used to extract features from the received data, and obtain the task relevance indicators of each brain region combination through an activation function, sort the task relevance indicators in descending order, and obtain a preset quantity of task-related brain region combinations; The brain region classification prediction layer is used to receive the fusion features corresponding to the task-related brain region combinations and obtain the brain region-related classification results.

10. The multi-parameter fusion mood disorder assessment system based on a temporal dynamic graph network according to claim 9, wherein, The observation time unit gating layer includes an input layer, an observation time sparse gating layer, and an observation time unit classification prediction layer; The input layer is used to obtain the alternative EEG features related to the length of each observation time unit based on the task-related brain region combinations and the task-related frequency bands, flatten the alternative EEG features related to the length of each observation time unit, splice them with the demographic coding features respectively, and input them into the observation time gating layer; The output node quantity of the observation time sparse gating layer is the quantity of observation time unit length categories, which is used to extract features from the received data, and obtain the task relevance indicators of the lengths of each observation time unit through an activation function, and select the observation time unit length with the largest task relevance indicator as the task-related observation time unit; The observation time unit classification prediction layer is used to receive the temporal dynamic graph network features corresponding to the task-related observation time units and the alternative subclass parameter combinations, and obtain the observation time unit-related classification results.

Images