A multi-parameter fusion mood disorder assessment system based on temporal dynamic graph network

Abstract

The present invention provides a multi-parameter fusion mood disorder assessment system based on a time-series dynamic graph network, comprising: a data acquisition unit for collecting EEG signals of patients to be assessed; an EEG encoding unit for extracting EEG features corresponding to the EEG signals; a time-series dynamic graph network feature extraction unit for constructing a time-series dynamic graph network sequence related to multiple EEG parameter combinations and extracting related time-series dynamic graph network features; a demographic information encoding unit for encoding demographic information to obtain demographic encoding features; and a hierarchical multi-level gating unit for predicting mood disorder assessment results based on the time-series dynamic graph network features corresponding to the EEG parameter combinations obtained by screening the obtained time-series dynamic graph network features of the sequentially arranged multi-level gating layers. The present invention solves the problem that the mood disorder assessment system in the prior art does not consider the correlation between the multi-parameter combination and the task, and does not screen the brain network model through hierarchical gating, resulting in limited assessment accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of clinical medicine technology, and in particular relates to a multi-parameter fusion mood disorder assessment system based on a time-series dynamic graph network. Background Art

[0002] With the development of society, the high incidence of depression and bipolar disorder has attracted widespread attention from the society. The work of psychiatrists in differentiating normal / depressed mood and unipolar depression / bipolar depression is becoming increasingly heavy, and the differentiation of unipolar depression / bipolar depression is a difficult point in clinical differentiation work.

[0003] Electroencephalography (EEG) is a commonly used method for recording electrical activity generated by the cortex. It is characterized by high resolution, non-invasiveness, and low cost. It can reflect the neural activity of various functional areas of the human brain. Therefore, it has applications in brain disease diagnosis, motor recovery, and neurological disease assessment. In recent years, in the fields of psychology and cognitive neuroscience, research on mood disorders based on EEG has also been on the rise.

[0004] However, current EEG-based mood disorder assessment systems are usually based on raw EEG signals and are easily affected by EEG signal noise. In addition, the modeling process lacks dynamic modeling of brain networks and feature decoupling of frequency bands, brain regions, and observation time scales, resulting in poor interpretability and generalization of the assessment system. Summary of the Invention

[0005] In view of the above analysis, the present invention aims to provide a multi-parameter fusion mood disorder assessment system based on a time-series dynamic graph network, which is used to solve the problem that the mood disorder assessment system in the existing technology lacks dynamic modeling of the brain network during the modeling process, resulting in poor interpretability and generalization of the assessment system.

[0006] The purpose of the present invention is mainly achieved through the following technical solutions:

[0007] The present invention provides a multi-parameter fusion mood disorder assessment system based on a time-series dynamic graph network, the system comprising:

[0008] A data acquisition unit, used to collect EEG signals of the patient to be evaluated;

[0009] an EEG encoding unit, configured to encode the EEG signal into a potential representation to obtain an EEG feature corresponding to the EEG signal;

[0010] A time series dynamic graph network feature extraction unit is used to construct a variety of EEG parameter combinations, establish time series dynamic graph network sequences of the EEG signals based on the EEG features corresponding to the various EEG parameter combinations, and perform feature extraction on each time series dynamic graph network sequence to obtain the time series dynamic graph network features related to each EEG parameter combination;

[0011] The demographic information encoding unit is used to encode and merge the demographic information of the patient to be evaluated and the state information when collecting EEG to obtain demographic coding features;

[0012] The hierarchical multi-level gating unit is used to screen the EEG characteristics and demographic coding characteristics through the multi-level gating layers set in sequence to obtain task-related EEG parameter combinations; and predict the mood disorder assessment results based on the time series dynamic graph network characteristics related to the screened EEG parameter combinations.

[0013] Furthermore, the parameters in the EEG parameter combination include frequency band, brain area and observation time unit.

[0014] Furthermore, the EEG parameter combination is constructed by the following method:

[0015] Combine different brain regions in pairs to obtain multiple brain region combinations;

[0016] Set multiple observation time units of different lengths;

[0017] Different frequency bands, brain area combinations and observation time units are combined in a preset order of levels to obtain multiple EEG parameter combinations and subclass parameter combinations corresponding to each parameter.

[0018] Furthermore, the temporal dynamic graph network feature extraction unit obtains the temporal dynamic graph network features related to each parameter combination through the dynamic edge convolution network and the temporal embedding module; wherein,

[0019] The dynamic edge convolutional network is used to construct a dynamic graph network sequence related to various EEG parameter combinations;

[0020] The temporal embedding module is used to learn the feature relationship of the dynamic graph network sequence through a long short-term memory network, and to extract features through a fully connected layer to obtain temporal dynamic graph network features.

[0021] Furthermore, the dynamic edge convolutional network constructs a dynamic graph network sequence related to each EEG parameter combination based on feature similarity by the following method:

[0022] Dividing the EEG signal into a plurality of observation windows according to the length of the observation time unit in the EEG parameter combination to be processed;

[0023] For each observation window, the channel dimension of the EEG feature corresponding to the EEG parameter combination is used as a node of the graph;

[0024] For each node, calculate its distance to other nodes using Euclidean distance, sort the distances in ascending order, select the n closest nodes as neighbors, and each neighbor node forms an edge with the node;

[0025] The features of the two nodes corresponding to each edge are concatenated to obtain the features of each edge, and the features of each edge are converted into the features of the target quantity through a multi-layer perceptron;

[0026] Aggregate the features of the target quantities of all edges corresponding to each node, and perform a global pooling operation on the aggregated features of all nodes to obtain the one-dimensional features corresponding to the observation window;

[0027] The one-dimensional features corresponding to all observation windows are spliced ​​in chronological order to obtain a dynamic graph network sequence related to the EEG parameter combination.

[0028] Furthermore, the long short-term memory network is composed of a plurality of sequential network units connected in series;

[0029] The temporal embedding module is used to learn the feature relationship of the dynamic graph network sequence through the long short-term memory network and perform feature extraction through the fully connected layer, including:

[0030] aligning each observation window in the EEG signal with each temporal network unit of the long short-term memory network;

[0031] The one-dimensional features corresponding to each observation window are input into the corresponding temporal network unit; the outputs of multiple temporal network units are flattened to one dimension and input into the fully connected layer to extract the temporal dynamic graph network features.

[0032] Furthermore, the hierarchical multi-level gating unit includes a frequency band gating layer, a brain region combination gating layer, an observation time unit gating layer and a fusion evaluation layer;

[0033] The frequency band gating layer is used to screen out a preset number of task-related frequency bands based on the EEG features corresponding to different frequency bands; and to perform classification prediction based on the temporal dynamic graph network features related to the subclass parameter combinations corresponding to the task-related frequency bands to obtain frequency band-related evaluation results;

[0034] The brain region combination gating layer uses the screened task-related frequency bands as constraints to screen task-related brain region combinations, and performs classification prediction based on the temporal dynamic graph network features related to the subclass parameter combinations corresponding to the combination of the task-related brain region combinations and the frequency bands, to obtain brain region-related evaluation results;

[0035] The observation time unit gating layer uses the screened task-related frequency bands and brain region combinations as constraints to screen task-related observation time units, and performs classification prediction based on the temporal dynamic graph network features related to the task-related frequency bands, brain region combinations, and parameter combinations corresponding to the observation time units to obtain observation time-related evaluation results;

[0036] The fusion evaluation layer is used to perform weighted fusion on the frequency band related evaluation results, the brain region related evaluation results and the observation time related evaluation results to obtain a final mood disorder evaluation result.

[0037] Furthermore, the band gating layer includes an input layer, a band sub-level dense gating layer, a band sparse gating layer and a band classification prediction layer;

[0038] The input layer is used to splice the EEG features corresponding to each frequency band with the demographic coding features, and input them into the frequency band sub-level dense gating layer and the frequency band sparse gating layer respectively;

[0039] The number of outputs of the frequency band sub-level dense gating layer is the same as the number of sub-class parameter combinations corresponding to each frequency band, and is used to extract features from the input data and obtain weights corresponding to each sub-class parameter combination through an activation function; based on the weights, the time series dynamic graph network features corresponding to each sub-class parameter combination corresponding to each frequency band are weightedly fused to obtain fused features corresponding to each frequency band;

[0040] The number of outputs of the frequency band sparse gating layer is the same as the number of frequency bands, and is used to extract features from the input data, and obtain a task relevance index for each frequency band through an activation function, and sort the task relevance indexes in descending order to screen a preset number of task-related frequency bands;

[0041] The frequency band classification prediction layer is used to receive fusion features corresponding to task-related frequency bands and predict frequency band-related classification results.

[0042] Furthermore, the brain region combination gating layer includes an input layer, a brain region sub-level dense gating layer, a brain region sparse gating layer and a brain region classification prediction layer;

[0043] The input layer is used to flatten the channel dimension and feature dimension of the task-related frequency band corresponding to each brain region combination, and splice them with the demographic coding features, and input them into the brain region sub-level dense gating layer and the brain region sparse gating layer respectively;

[0044] The number of outputs of the brain region sub-level dense gating layer is the number of candidate parameter combinations consisting of task-related frequency bands and observation time units corresponding to each brain region combination; the brain region sub-level dense gating layer is used to extract features from the received data and obtain weights of each candidate parameter combination corresponding to each brain region combination through an activation function; and based on the weights, the time series dynamic graph network features of each candidate parameter combination corresponding to each brain region combination are weightedly fused to obtain fused features corresponding to each brain region combination;

[0045] The number of output layers of the brain region sparse gating layer is the number of brain region combinations, which is used to extract features from the received data, and obtain the task relevance index of each brain region combination through the activation function, and arrange the task relevance indexes in descending order to obtain a preset number of task-related brain region combinations;

[0046] The brain region classification prediction layer is used to receive fusion features corresponding to task-related brain region combinations and obtain brain region-related classification results.

[0047] Furthermore, the observation time unit gating layer includes an input layer, an observation time sparse gating layer and an observation time unit classification prediction layer;

[0048] The input layer is used to obtain candidate EEG features related to the length of each observation time unit based on the task-related brain area combination and the task-related frequency band, flatten the candidate EEG features related to the length of each observation time unit, and splice them with the demographic coding features respectively, and input them into the observation time gating layer;

[0049] The number of output nodes of the observation time sparse gating layer is the number of observation time unit length categories, which is used to extract features from the received data, and obtain the task relevance index of each observation time unit length through the activation function, and select the observation time unit length with the largest task relevance index as the task-related observation time unit;

[0050] The observation time unit classification prediction layer is used to receive the time series dynamic graph network features corresponding to the task-related observation time units and the candidate subclass parameter combinations, and obtain the observation time unit related classification results.

[0051] Beneficial effects of this technical solution:

[0052] 1. This paper proposes an EEG signal analysis technology based on a time-series dynamic graph network that models EEG features across multiple frequency bands, multiple brain regions, and multiple observation time units. This technology also incorporates subject demographic information into a hierarchical gating model, screening task-related parameters based on the frequency band, brain region, and observation time unit dimensions. The resulting parameter features are then weighted and fused for use in mood disorder assessment. This improves the accuracy and efficiency of mood disorder assessment, making it particularly suitable for diagnosing mood disorders across multiple subjects. The assessment system exhibits excellent generalizability and accuracy.

[0053] 2. The present invention selects brain regions related to mood disorders, groups them in pairs, and combines frequency bands and the length of observation time units to construct assessment data. Based on the latent features of EEG signals and the linkages between brain regions, as well as between frequency bands and observation time, it can accurately extract features related to mood disorders for use in mood disorder assessment, greatly improving the accuracy of mood disorder assessment.

[0054] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description or be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. Throughout the drawings, the same reference symbols denote the same components.

[0056] Figure 1 2. It is a schematic diagram of a multi-parameter fusion mood disorder assessment system based on a time-series dynamic graph network according to an embodiment of the present invention;

[0057] Figure 2 Schematic diagram of the EEG coding model according to an embodiment of the present invention;

[0058] Figure 3 Schematic diagram of a network feature of a time series dynamic graph constructed based on a combination of multiple EEG parameters according to an embodiment of the present invention;

[0059] Figure 4 This is a model training flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the implementation cases of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0061] One embodiment of the present invention provides a multi-parameter fusion mood disorder assessment system based on a time series dynamic graph network. Figure 1 As shown, the system includes:

[0062] A data acquisition unit, used to collect EEG signals of the patient to be evaluated;

[0063] an EEG encoding unit, configured to encode the EEG signal into a potential representation to obtain an EEG feature corresponding to the EEG signal;

[0064] A time series dynamic graph network feature extraction unit is used to construct a variety of EEG parameter combinations, establish time series dynamic graph network sequences of the EEG signals based on the EEG features corresponding to the various EEG parameter combinations, and perform feature extraction on each time series dynamic graph network sequence to obtain the time series dynamic graph network features related to each parameter combination;

[0065] The demographic information encoding unit is used to encode and merge the demographic information of the patient to be evaluated and the state information when collecting EEG to obtain demographic coding features;

[0066] The hierarchical multi-level gating unit is used to screen the EEG characteristics and demographic coding characteristics through the multi-level gating layers set in sequence to obtain task-related EEG parameter combinations; and predict the mood disorder assessment results based on the time series dynamic graph network characteristics corresponding to the screened EEG parameter combinations.

[0067] Specifically, the data acquisition unit of this embodiment uses a resting-state multi-channel scalp EEG acquisition device to collect the subject's electrophysiological signals for subsequent EEG signal analysis and mood disorder assessment.

[0068] After collecting the EEG signal, it is first preprocessed by the data preprocessing unit, specifically including downsampling, re-referencing, notching to remove power frequency, and bandpass filtering. The ICA method is used to remove electro-oculogram (EOG), electrocardiogram (ECG), and myoelectricity (EMG), and abnormal amplitude segments are removed. The processed segments are then divided into EEG segments of fixed time length. Removing abnormal amplitude segments includes: first time slicing the EEG data after bandpass filtering, then calculating the amplitude within the time slice, and deleting time slices whose amplitude is not within the preset range. In this embodiment, the EEG segment time length is set to 120 seconds, and the evaluation results of all EEG segments are averaged as the final evaluation result.

[0069] After data preprocessing of the EEG signal, each EEG segment of the EEG signal is encoded into a potential representation through the EEG encoding unit and brain regions are grouped:

[0070] Specifically, such as Figure 2As shown in the figure, each 1-second single-channel EEG signal is encoded into a latent representation using a pre-trained EEG encoding model. Each latent representation has a corresponding feature domain for each EEG frequency band. EEG signals corresponding to different channels are grouped based on the electrode location and brain region delineation of the EEG signal.

[0071] This embodiment uses the existing EEG encoding model (VAEEG) to encode a 1-second single-channel EEG signal into a latent representation space. Each preprocessed signal is converted into characteristic signals represented by different frequency bands, such as delta[1,4]Hz, theta[4-8Hz], alpha[8-13]Hz, beta1[13-25]Hz, and beta2[25-40]Hz. The representation space dimensions corresponding to each frequency band in each 1-second segment are delta_zim, theta_zim, alpha_zim, beta1_zim, and beta2_zim. Corresponding to each different electrode position, the brain is divided into functional regions: frontal, parietal, central, left temporal, right temporal, and occipital, and the latent representation corresponding to each brain region is divided into the corresponding region.

[0072] This embodiment uses the EEG encoding unit to extract stable features from complex EEG signals by reducing the dimensionality of the EEG signals, thereby achieving a certain denoising effect.

[0073] Among them, in order to divide the observation time unit in the next step, the EEG segment for feature extraction in this step should be as short as possible, or the observation time unit should be used directly for feature extraction.

[0074] Furthermore, the temporal dynamic graph network feature extraction unit constructs an EEG parameter combination by the following method:

[0075] Combine different brain regions in pairs to obtain multiple brain region combinations;

[0076] Set multiple observation time units of different lengths;

[0077] Different frequency bands, brain area combinations and observation time units are combined in a preset level order to obtain multiple EEG parameter combinations and sub-category parameter combinations corresponding to each parameter, that is, the sum of the sub-category parameter combinations corresponding to each parameter constitutes all EEG parameter combinations.

[0078] Specifically, for frequency bands, this embodiment selects five frequency bands with a relatively high correlation with mood disorders: delta, theta, alpha, beta1, and beta2; for brain regions, the selected regions are frontal, parietal, central, left temporal, right temporal, and occipital; for the length of the observation time unit, three lengths are preset: short, medium, and long, corresponding to observation time unit lengths of 5s, 10s, and 15s, respectively; this embodiment uses the frequency band class as the first-level parameter, the brain region class as the second-level parameter, and the observation time class as the third-level parameter; a fixed level order is used to combine different parameters to obtain an EEG parameter combination, such as: ;

[0079] Among them, the number of EEG parameter combinations obtained is:

[0080] ;

[0081] The brain area combinations are frontal area-parietal area, frontal area-central area, etc. The number of brain area combinations obtained is: ;

[0082] For each level of parameters, there is a corresponding sub-category parameter combination. For example, for the delta band, the number of sub-category parameter combinations is:

[0083] ;

[0084] like: .

[0085] Furthermore, the temporal dynamic graph network feature extraction unit obtains temporal dynamic graph network features related to each parameter combination through a dynamic edge convolutional network and a temporal embedding module;

[0086] The dynamic edge convolutional network is used to construct a dynamic graph network sequence related to various EEG parameter combinations;

[0087] The temporal embedding module is used to learn the feature relationship of the dynamic graph network sequence through a long short-term memory network, and to extract features through a fully connected layer to obtain temporal dynamic graph network features.

[0088] Specifically, the dynamic edge convolutional network constructs a dynamic graph network sequence related to each EEG parameter combination based on feature similarity by the following method:

[0089] Dividing the EEG signal into a plurality of observation windows according to the length of the observation time unit in the EEG parameter combination to be processed;

[0090] For each observation window, the channel dimension of the EEG feature corresponding to the EEG parameter combination is used as a node of the graph;

[0091] For each node, calculate its distance to other nodes using Euclidean distance, sort the distances in ascending order, select the n closest nodes as neighbors, and each neighbor node forms an edge with the node;

[0092] The features of the two nodes corresponding to each edge are concatenated to obtain the features of each edge, and the features of each edge are converted into the features of the target quantity through a multi-layer perceptron;

[0093] Aggregate the features of the target quantities of all edges corresponding to each node, and perform a global pooling operation on the aggregated features of all nodes to obtain the one-dimensional features corresponding to the observation window;

[0094] The one-dimensional features corresponding to all observation windows are spliced ​​in chronological order to obtain a dynamic graph network sequence related to the EEG parameter combination.

[0095] This embodiment is different from the static brain network analysis based on anatomical connections and functional connections. This embodiment adopts a completely data-driven approach to dynamically construct a graph structure based on the similarity of features in each time window. The K-nearest neighbor graph construction method can be used to construct a dynamic graph for each observation window. For each observation window, the corresponding EEG feature input dimension is (number of channels, number of features), where the number of channels is the number of all channels contained in the corresponding brain region combination. The channel dimension of the EEG feature is used as the node of the graph. For each node, the distance between it and other nodes is calculated by Euclidean distance. The distances are arranged in ascending order, and the top 4 nearest nodes are selected as neighbors. Each node forms an edge with the node to construct the graph structure of each observation window.

[0096] The graph structure is then subjected to feature aggregation and conversion, including taking the features of the two nodes connecting each edge as new features, and connecting a nonlinear multilayer perceptron to convert the new features into features of the target quantity; in order to reduce the complexity of the model, the number of layers of the multilayer perceptron is 1; and the features of the target quantity corresponding to each edge are then message aggregated, that is, for each node, the features of the target quantities corresponding to all edges are aggregated. The aggregation method adopted in the present invention is mean aggregation.

[0097] After message aggregation for each node, the features of all nodes in each observation window are averaged and pooled, so that the graph structure features of each observation window are compressed into one-dimensional features, represented as EdgeConv_zim; for fixed frequency band and brain area combinations, we only need to focus on the internal dynamic changes between specific frequency bands and specific brain areas. Corresponding to each observation window, a graph structure will be obtained. This graph structure is a dynamic graph that adaptively aggregates edge nodes. The graph structures of multiple observation windows are spliced ​​together to form a dynamic graph network sequence.

[0098] Preferably, the long short-term memory network is composed of a plurality of sequential network units connected in series;

[0099] The temporal embedding module is used to learn the feature relationship of dynamic graph network sequences through the long short-term memory network and perform feature extraction through the fully connected layer, such as Figure 3 Shown, including:

[0100] aligning each observation window in the EEG signal with each temporal network unit of the long short-term memory network;

[0101] The one-dimensional features corresponding to each observation window are input into the corresponding temporal network unit; the outputs of multiple temporal network units are flattened to one dimension and input into the fully connected layer to extract the temporal dynamic graph network features.

[0102] The present invention uses a long short-term memory network to learn the relationship between dynamic graph network features. The network is composed of a series of time series network units (LSTM cells) with the number of observation windows, where the time series information transmission direction is single-phase, the hidden layer of each time series network unit is 1, and the output dimension is set to LSTM_zim; the features of each observation window are aligned with the time series network unit (LSTMcell), and the output EdgeConv_zim of the dynamic edge convolution network is used as the input of the corresponding time series embedding module; then the outputs of multiple time series network units are flattened to one dimension, and the dimension is obtained. The feature of length; the obtained feature connection target is output as the fully connected layer of EdgeLstm_zim, and the temporal dynamic graph network features are extracted.

[0103] The temporal dynamic graph network feature extraction model is obtained by iterative training of a pre-constructed training sample set;

[0104] When constructing a training sample set, if applied to depression / non-depression assessment, EEG features corresponding to multiple mood disorder patients and healthy subjects are obtained, and normal or mood disorder labels are marked; if applied to unipolar / bipolar depression assessment, EEG features corresponding to multiple unipolar depression and bipolar depression patients are obtained, and unipolar depression or bipolar depression labels are marked to construct a training sample set;

[0105] When training the temporal dynamic graph feature extraction model, each EEG feature in the training sample set was grouped according to each EEG parameter combination, resulting in a training set corresponding to each EEG parameter combination. The temporal dynamic graph network feature extraction model was then connected to a classifier consisting of two multilayer perceptrons. Using each training set, the model parameters were optimized using cross-entropy as the loss function. A validation set was used to evaluate the discriminant performance. If both the sensitivity and specificity of the validation set were above a certain threshold, the pre-trained model was saved. Otherwise, the model parameters were adjusted within a preset number of times until the validation set reached the required threshold. If the threshold was exceeded, the model was disregarded from the subsequent evaluation model. The model parameters obtained from each training set were saved, resulting in a dynamic graph network feature extraction model for each EEG parameter combination.

[0106] The demographic information encoding unit obtains the demographic coding features by the following method:

[0107] Read the subject's gender, age, and eye-opening and eye-closing information when collecting EEG signals, and perform preprocessing using the following methods:

[0108] For age information, standardization is used to unify the numerical ranges of different features to avoid large gradient differences during model learning. This can be achieved using standardization (x-x_mean), x-std, or normalization methods. For gender information, one-hot encoding is used, encoding males as (0, 1) and females as (1, 0). For eye-opening and eye-closing information, one-hot encoding is used, encoding eyes-open as (0, 1) and eyes-closed as (1, 0).

[0109] For each dimension of preprocessed information, the corresponding information encoding module consisting of multiple fully connected layers is input and nonlinear transformation is performed using activation functions;

[0110] The information obtained after nonlinear transformation is encoded and merged, and mapped to a unified representation space to obtain demographic coding features.

[0111] Furthermore, the hierarchical multi-level gating unit includes a frequency band gating layer, a brain region combination gating layer, an observation time unit gating layer and a fusion evaluation layer;

[0112] The frequency band gating layer is used to screen out a preset number of task-related frequency bands based on the EEG features corresponding to different frequency bands; and to perform classification prediction based on the temporal dynamic graph network features related to the subclass parameter combinations corresponding to the task-related frequency bands to obtain frequency band-related evaluation results;

[0113] The brain region combination gating layer uses the screened task-related frequency bands as constraints to screen task-related brain region combinations, and performs classification prediction based on the temporal dynamic graph network features related to the subclass parameter combinations corresponding to the combination of the task-related brain region combinations and the frequency bands, to obtain brain region-related evaluation results;

[0114] The observation time unit gating layer uses the screened task-related frequency bands and brain region combinations as constraints to screen task-related observation time units, and performs classification prediction based on the temporal dynamic graph network features related to the task-related frequency bands, brain region combinations, and parameter combinations corresponding to the observation time units to obtain observation time-related evaluation results;

[0115] The fusion evaluation layer is used to perform weighted fusion on the frequency band related evaluation results, the brain region related evaluation results and the observation time related evaluation results to obtain a final mood disorder evaluation result.

[0116] This embodiment is divided into multiple gating layers based on various parameters: the first layer is the frequency band gating layer, the second layer is the brain region gating layer, and the third layer is the observation time gating layer. The first gating layer is further divided into sub-layers for multi-band features, ranging from delta sub-gating layer to beta2 sub-gating layer. The second brain region gating layer is composed of any two brain networks, and is divided into sub-gating layers from frontal region to parietal region to left temporal region to occipital region. The third observation time gating layer does not require sub-gating layers.

[0117] The band gating layer includes an input layer, a band sub-level dense gating layer, a band sparse gating layer and a band classification prediction layer;

[0118] The input layer is used to splice the EEG features corresponding to each frequency band with the demographic coding features, and input them into the frequency band sub-level dense gating layer and the frequency band sparse gating layer respectively;

[0119] The frequency band sub-level dense gating layer and the frequency band sparse gating layer are both constructed based on a multi-layer perceptron;

[0120] The number of outputs of the frequency band sub-level dense gating layer is the same as the number of sub-class parameter combinations corresponding to each frequency band, and is used to extract features from the input data and obtain weights corresponding to each sub-class parameter combination through an activation function; based on the weights, the time series dynamic graph network features corresponding to each sub-class parameter combination corresponding to each frequency band are weightedly fused to obtain fused features corresponding to each frequency band;

[0121] The number of outputs of the frequency band sparse gating layer is the same as the number of frequency bands, and is used to extract features from the input data, and obtain a task relevance index for each frequency band through an activation function, and sort the task relevance indexes in descending order to screen a preset number of task-related frequency bands;

[0122] The frequency band classification prediction layer is used to receive fusion features corresponding to task-related frequency bands and predict frequency band-related classification results.

[0123] Specifically, for the frequency band sub-gating layer, since the preselected frequency bands are fixed, EEG features associated with the frequency band dimension are concatenated with demographic coding features. These features are then connected to a multilayer perceptron whose outputs are the same as the number of target hyperparameter combinations. After the output features are passed through a softmax layer, weights corresponding to the features of each parameter combination are obtained. Based on these weights, the corresponding time-series dynamic graph network features are weightedly fused to obtain fused features for each frequency band, denoted as EdgeLstm_delta, EdgeLstm_theta, EdgeLstm_alpha, EdgeLstm_beta1, and EdgeLstm_beta2. To ensure feature diversity in the initial encoding phase of the model and enhance stability, dense gating is used to selectively fuse features corresponding to sub-level parameters.

[0124] For the frequency band gating layer, the multilayer perceptron uses the number of frequency bands as the output dimension and selects the frequency bands whose output features are ranked as the top 2 after passing through the softmax layer. This embodiment uses sparse gating to select the target frequency band, with the aim of selecting the frequency band most relevant to the task objective.

[0125] The brain region combination gating layer includes an input layer, a brain region sub-level dense gating layer, a brain region sparse gating layer and a brain region classification prediction layer;

[0126] The input layer is used to flatten the channel dimension and feature dimension of the task-related frequency band corresponding to each brain region combination, and splice them with the demographic coding features, and input them into the brain region sub-level dense gating layer and the brain region sparse gating layer respectively;

[0127] The brain region sub-level dense gating layer and the brain region sparse gating layer are both obtained based on multi-layer perceptron training;

[0128] The number of outputs of the brain region sub-level dense gating layer is the number of candidate parameter combinations consisting of task-related frequency bands and observation time units corresponding to each brain region combination; the brain region sub-level dense gating layer is used to extract features from the received data and obtain weights of each candidate parameter combination corresponding to each brain region combination through an activation function; and based on the weights, the time series dynamic graph network features of each candidate parameter combination corresponding to each brain region combination are weightedly fused to obtain fused features corresponding to each brain region combination;

[0129] The number of output layers of the brain region sparse gating layer is the number of brain region combinations, which is used to extract features from the received data, and obtain the task relevance index of each brain region combination through the activation function, and arrange the task relevance indexes in descending order to obtain a preset number of task-related brain region combinations;

[0130] The brain region classification prediction layer is used to receive fusion features corresponding to task-related brain region combinations and obtain brain region-related classification results.

[0131] The observation time unit gating layer includes an input layer, an observation time sparse gating layer and an observation time unit classification prediction layer;

[0132] The input layer is used to obtain candidate EEG features related to the length of each observation time unit based on the task-related brain area combination and the task-related frequency band, flatten the candidate EEG features related to the length of each observation time unit, and splice them with the demographic coding features respectively, and input them into the observation time gating layer;

[0133] The number of output nodes of the observation time sparse gating layer is the number of observation time unit length categories, which is used to extract features from the received data, and obtain the task relevance index of each observation time unit length through the activation function, and select the observation time unit length with the largest task relevance index as the task-related observation time unit;

[0134] The observation time unit classification prediction layer is used to receive the time series dynamic graph network features corresponding to the task-related observation time units and the candidate subclass parameter combinations, and obtain the observation time unit related classification results.

[0135] Specifically, for the brain region combination sub-level gating layer, the frequency band selected in the frequency band gating layer is used to constrain the alternative models of the gating module of this layer. Since the pre-selected brain region selection is fixed, it is necessary to select the electrodes corresponding to the selected brain region in the channel dimension of each frequency band, flatten the channel part and the feature part of the EEG feature, and splice it with the demographic coding feature. The input is connected to a multi-layer sensor with the number of alternative models of this layer as the output, and then the output is passed through the softmax layer to obtain the weight of each alternative model. According to the weight, the corresponding time series dynamic graph network features are weighted and fused to obtain the fusion features corresponding to each brain region combination. In order to enhance the stability of the model and the diversity of features, this embodiment adopts dense gating to fuse the features corresponding to the sub-level parameters.

[0136] For the brain region combination gating layer, the multilayer perceptron uses the number of brain regions as the output dimension, selects the output features and sorts them into the corresponding frequency bands of the top 2 after passing through the softmax layer; this embodiment uses sparse gating to select the target brain region, with the goal of selectively selecting a temporal dynamic graph network feature extraction model related to the task goal.

[0137] For the observation time unit gating layer, its parameters are constrained by task-relevant frequency bands and brain regions, without involving sub-level gating layers. The candidate EEG features associated with each observation time unit length are flattened and concatenated with the demographic coding features. This is then connected to a multilayer perceptron with the observation time unit parameter as the number of output nodes. The output features are then sorted by the top 1 observation time unit length after passing through a softmax layer. Sparse gating is used here to select observation time unit lengths, aiming to selectively select task-relevant observation time units.

[0138] The hierarchical multi-level gating unit is trained using a training sample set constructed during the training of the temporal dynamic graph network feature extraction model, and temporal dynamic graph network features of each sample data associated with each parameter combination obtained during the training of the temporal dynamic graph network feature extraction model;

[0139] The specific training process is as follows Figure 4 As shown, during the training of the hierarchical multi-level gating unit, redundant data of the temporal dynamic graph network features related to each parameter combination obtained by the temporal dynamic graph network feature extraction unit is hierarchically deleted to obtain an alternative temporal dynamic graph network feature set for iterative optimization of the model, including:

[0140] The data in the training sample set is divided into K folds, and the training data in the K folds are iteratively used to obtain the temporal dynamic graph network features related to all EEG parameter combinations;

[0141] The number of samples is fixed, and the correlation between each parameter combination is calculated based on the network characteristics of the time series dynamic graph; the mean of the correlation is calculated in the sample dimension, and the parameter combination whose absolute value of the correlation is greater than a first preset threshold is marked;

[0142] After K iterations, the parameter combinations whose number of tags is greater than the second preset threshold are removed to obtain an alternative temporal dynamic graph network feature set.

[0143] For each level of subclass parameter combination, the features of different temporal dynamic graph network encodings may have high correlation. The present invention introduces a method to reduce model redundancy in this step so that the subsequent gating network can achieve better weight distribution.

[0144] Specifically, the training set is divided into K folds. The training data from each K fold is iteratively used to obtain the temporal dynamic graph network features related to all parameter combinations. The feature dimension of each batch is (number of samples, number of hyperparameter combinations, EdgeLstm_zim). With a fixed number of samples, the correlations between each parameter combination are calculated based on the features. The mean of the correlations is calculated along the sample dimension, resulting in a feature dimension of (number of parameter combinations, number of parameter combinations). Dimensions with an absolute correlation value greater than 0.65 are marked. After iterating this step K times, dimensions with a marked value greater than 0.8*K are removed. Finally, a set of candidate temporal dynamic graph network models is obtained.

[0145] During the training process, this embodiment connects the dynamic graph network features corresponding to the frequency band gating layer, the brain network combination gating layer, and the observation time unit gating layer to the multilayer perceptron output classification results, and calculates the cross entropy of the output results of each layer. Different weights are set for each cross entropy, and the overall loss function of the model is defined as:

[0146] ;

[0147] ;

[0148] ;

[0149] ;

[0150] When performing weighted fusion, the weights of each gating layer must meet the following constraints:

[0151] ;

[0152] in, is the total loss of the model, is the band gating layer weight, is the brain region gating layer weight, is the gating layer weight of the time observation unit, is the cross entropy loss of the bandgating layer, is the cross entropy loss of the brain region gating layer, is the cross entropy loss of the gating layer of the temporal observation unit, To mark the label, is the predicted value of the band gating layer, is the predicted value of the brain region gating layer, is the predicted value of the gating layer of the observation time unit;

[0153] Since the model design is to progressively design the gating mechanism from the frequency band layer to the brain network combination layer to the observation time unit layer and realize the feature selection of the dynamic graph network, the principle is to achieve better mood disorder assessment through progressive feature angles and continuously refined feature backgrounds. Therefore, when setting the cross-entropy weight, the weight of each layer should be maintained: observation time unit layer weight > brain network combination layer weight > frequency band layer weight.

[0154] In summary, embodiments of the present invention provide a multi-parameter fusion mood disorder assessment system based on a temporal dynamic graph network. This system uses a hierarchical, multi-level gating mechanism to implement an expert model screening method that adapts to subject demographic information. This system also uses a method for progressively modeling a gated network based on multi-angle parameters (including frequency bands, brain region combinations, and observation time unit lengths). For each major parameter category (each layer), a sub-level gate is established for that layer. Dense modeling is used for gating at each sub-level to increase the diversity of sub-level features, while sparse modeling is used for deep gating to enhance the specificity of task discrimination. The present invention also employs a two-stage method for training a temporal dynamic graph convolutional network and a gated network. During pre-training of the spatiotemporal dynamic graph network, subject demographic information is not incorporated, and only EEG representations are learned. During the establishment of the gated network, demographic information is incorporated to assist in the selection of multi-level candidate graph network features. Furthermore, the present invention utilizes a temporal dynamic edge convolutional neural network model to achieve feature extraction, resulting in a stronger ability to extract complex brain function features. The system of the present invention significantly improves the accuracy of mood disorder assessment.

[0155] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0156] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A multi-parameter fusion mood disorder assessment system based on a time-series dynamic graph network, characterized by: include: A data acquisition unit, used to collect EEG signals of the patient to be evaluated; an EEG encoding unit, configured to encode the EEG signal into a potential representation to obtain an EEG feature corresponding to the EEG signal; A time series dynamic graph network feature extraction unit is used to construct a plurality of EEG parameter combinations, establish a time series dynamic graph network sequence of the EEG signal based on the EEG features corresponding to the various EEG parameter combinations, and perform feature extraction on each time series dynamic graph network sequence to obtain a time series dynamic graph network feature related to each EEG parameter combination; the parameters in the EEG parameter combination include frequency bands, brain regions, and observation time units; the EEG parameter combination is constructed by the following method: combining different brain regions in pairs to obtain a plurality of brain region combinations; setting a plurality of observation time units of different lengths; and combining different frequency bands, brain region combinations, and observation time units in a preset order of levels to obtain a plurality of EEG parameter combinations and subclass parameter combinations corresponding to each parameter; The demographic information encoding unit is used to encode and merge the demographic information of the patient to be evaluated and the state information when collecting EEG to obtain demographic coding features; A hierarchical multi-level gating unit is configured to screen the EEG features and demographic coding features through sequentially arranged multi-level gating layers to obtain task-related EEG parameter combinations; and to predict mood disorder assessment results based on the temporal dynamic graph network features related to the screened EEG parameter combinations; the hierarchical multi-level gating unit includes a frequency band gating layer, a brain region combination gating layer, an observation time unit gating layer, and a fusion assessment layer; the frequency band gating layer is configured to obtain frequency band-related assessment results; and the brain region combination gating layer is configured to obtain brain region-related assessment results; The observation time unit gating layer is used to obtain the observation time related evaluation results; the fusion evaluation layer is used to perform weighted fusion on the frequency band related evaluation results, brain region related evaluation results and observation time related evaluation results to obtain the final mood disorder evaluation results.

2. The multi-parameter fusion mood disorder assessment system based on time-series dynamic graph network according to claim 1 is characterized in that: The temporal dynamic graph network feature extraction unit obtains the temporal dynamic graph network features related to each parameter combination through the dynamic edge convolution network and the temporal embedding module; wherein, The dynamic edge convolutional network is used to construct a dynamic graph network sequence related to various EEG parameter combinations; The temporal embedding module is used to learn the feature relationship of the dynamic graph network sequence through a long short-term memory network, and to extract features through a fully connected layer to obtain temporal dynamic graph network features.

3. The multi-parameter fusion mood disorder assessment system based on time-series dynamic graph network according to claim 2 is characterized in that: The dynamic edge convolutional network constructs a dynamic graph network sequence related to each EEG parameter combination based on feature similarity through the following method: Dividing the EEG signal into a plurality of observation windows according to the length of the observation time unit in the EEG parameter combination to be processed; For each observation window, the channel dimension of the EEG feature corresponding to the EEG parameter combination is used as a node of the graph; For each node, calculate its distance to other nodes using Euclidean distance, sort the distances in ascending order, select the n closest nodes as neighbors, and each neighbor node forms an edge with the node; The features of the two nodes corresponding to each edge are concatenated to obtain the features of each edge, and the features of each edge are converted into the features of the target quantity through a multi-layer perceptron; Aggregate the features of the target quantities of all edges corresponding to each node, and perform a global pooling operation on the aggregated features of all nodes to obtain the one-dimensional features corresponding to the observation window; The one-dimensional features corresponding to all observation windows are spliced ​​in chronological order to obtain a dynamic graph network sequence related to the EEG parameter combination.

4. The multi-parameter fusion mood disorder assessment system based on time-series dynamic graph network according to claim 3 is characterized in that: The long short-term memory network is composed of a plurality of sequential network units connected in series; The temporal embedding module is used to learn the feature relationship of the dynamic graph network sequence through the long short-term memory network and perform feature extraction through the fully connected layer, including: aligning each observation window in the EEG signal with each temporal network unit of the long short-term memory network; The one-dimensional features corresponding to each observation window are input into the corresponding temporal network unit; the outputs of multiple temporal network units are flattened to one dimension and input into the fully connected layer to extract the temporal dynamic graph network features.

5. The multi-parameter fusion mood disorder assessment system based on time-series dynamic graph network according to claim 4 is characterized in that: The frequency band gating layer is used to screen out a preset number of task-related frequency bands based on the EEG features corresponding to different frequency bands; and to perform classification prediction based on the temporal dynamic graph network features related to the subclass parameter combinations corresponding to the task-related frequency bands to obtain frequency band-related evaluation results; The brain region combination gating layer uses the screened task-related frequency bands as constraints to screen task-related brain region combinations, and performs classification prediction based on the temporal dynamic graph network features related to the subclass parameter combinations corresponding to the combination of the task-related brain region combinations and the frequency bands, to obtain brain region-related evaluation results; The observation time unit gating layer uses the screened task-related frequency bands and brain area combinations as constraints to screen out task-related observation time units, and performs classification prediction based on the temporal dynamic graph network features related to the task-related frequency bands, brain area combinations and parameter combinations corresponding to the observation time units to obtain observation time-related evaluation results.

6. The multi-parameter fusion mood disorder assessment system based on time-series dynamic graph network according to claim 5 is characterized in that: The band gating layer includes an input layer, a band sub-level dense gating layer, a band sparse gating layer and a band classification prediction layer; The input layer is used to splice the EEG features corresponding to each frequency band with the demographic coding features, and input them into the frequency band sub-level dense gating layer and the frequency band sparse gating layer respectively; The number of outputs of the frequency band sub-level dense gating layer is the same as the number of sub-class parameter combinations corresponding to each frequency band, and is used to extract features from the input data and obtain the weight corresponding to each sub-class parameter combination through an activation function; Based on the weights, weighted fusion is performed on the time series dynamic graph network features corresponding to each subclass parameter combination corresponding to each frequency band to obtain a fusion feature corresponding to each frequency band; The number of outputs of the frequency band sparse gating layer is the same as the number of frequency bands, and is used to extract features from the input data, and obtain a task relevance index for each frequency band through an activation function, and sort the task relevance indexes in descending order to screen a preset number of task-related frequency bands; The frequency band classification prediction layer is used to receive fusion features corresponding to task-related frequency bands and predict frequency band-related classification results.

7. The multi-parameter fusion mood disorder assessment system based on time series dynamic graph network according to claim 6 is characterized in that: The brain region combination gating layer includes an input layer, a brain region sub-level dense gating layer, a brain region sparse gating layer and a brain region classification prediction layer; The input layer is used to flatten the channel dimension and feature dimension of the task-related frequency band corresponding to each brain region combination, and splice them with the demographic coding features, and input them into the brain region sub-level dense gating layer and the brain region sparse gating layer respectively; The number of outputs of the brain region sub-level dense gating layer is the number of candidate parameter combinations consisting of task-related frequency bands and observation time units corresponding to each brain region combination; the brain region sub-level dense gating layer is used to extract features from the received data and obtain weights of each candidate parameter combination corresponding to each brain region combination through an activation function; and based on the weights, the time series dynamic graph network features of each candidate parameter combination corresponding to each brain region combination are weightedly fused to obtain fused features corresponding to each brain region combination; The number of output layers of the brain region sparse gating layer is the number of brain region combinations, which is used to extract features from the received data, and obtain the task relevance index of each brain region combination through the activation function, and arrange the task relevance indexes in descending order to obtain a preset number of task-related brain region combinations; The brain region classification prediction layer is used to receive fusion features corresponding to task-related brain region combinations and obtain brain region-related classification results.

8. The multi-parameter fusion mood disorder assessment system based on time-series dynamic graph network according to claim 7 is characterized in that: The observation time unit gating layer includes an input layer, an observation time sparse gating layer and an observation time unit classification prediction layer; The input layer is used to obtain candidate EEG features related to the length of each observation time unit based on the task-related brain area combination and the task-related frequency band, flatten the candidate EEG features related to the length of each observation time unit, and then splice them with the demographic coding features respectively, and input them into the observation time sparse gating layer; The number of output nodes of the observation time sparse gating layer is the number of observation time unit length categories, which is used to extract features from the received data, and obtain the task relevance index of each observation time unit length through the activation function, and select the observation time unit length with the largest task relevance index as the task-related observation time unit; The observation time unit classification prediction layer is used to receive the time series dynamic graph network features corresponding to the task-related observation time units and the candidate subclass parameter combinations, and obtain the observation time unit related classification results.

Images