A dual-collaborative learning brain network feature classification system and a training method thereof

Abstract

The present application relates to the field of medical image analysis and computer-aided diagnosis of brain diseases, and proposes a dual cooperative learning brain network feature classification system and its training method, which specifically comprises: using an automatic anatomical labeling template, constructing static and dynamic functional brain networks based on the average time series signals of brain regions using Pearson correlation coefficients; using a dual-flow network to extract static and dynamic feature representations of the functional brain network; using a pruned and grafted Transformer module to dynamically adjust the features between the static and dynamic functional brain networks and their respective interiors; introducing a cooperative contrast loss function to learn high-level feature representations; and using a multi-layer perceptron to perform feature classification based on the output high-level features. The brain network feature classification system and its training method proposed by the present application not only significantly improve the feature classification performance, but also provide new insights into the interactive dynamics of brain activity.

Description

Technical Field

[0001] This invention relates to the fields of medical image analysis and computer-aided diagnosis of brain diseases, specifically to a brain network feature classification system with dual collaborative learning and its training method. Background Technology

[0002] Alzheimer's disease is one of the most common brain disorders among the elderly. To address this, researchers have proposed numerous computer-aided diagnostic methods based on functional brain networks constructed from rs-fMRI data.

[0003] To quantify neuronal activity in resting-state functional magnetic resonance imaging (rs-fMRI) time series, most previous functional connectivity studies have relied on the implicit assumption that neuronal activity is static during fMRI scans. The resulting brain networks, termed static functional brain networks, effectively reflect the spatial topological information of the brain. However, temporal variations in these networks, which may be closely related to cognitive function, are often overlooked. To capture the dynamics of brain networks, some recent studies have employed sliding time window techniques, dividing the time series into multiple subsequences to obtain dynamic functional brain networks. Dynamic functional brain networks can reveal temporal information but may also introduce noise that could affect functional connectivity.

[0004] In fact, static and dynamic functional connectivity, as two view representations of functional brain networks (FBNs), naturally combine their functional characteristics for brain disease diagnosis. For example, some researchers have proposed using fMRI and magnetoencephalography (MEG) data combined with static and dynamic functional network connectivity analysis to identify schizophrenia, achieving good results. However, this method extracts the features of the two views independently. To address this issue, some studies have attempted to use a convolutional neural network (CNN) framework to learn hierarchical complementary features through the construction of paired static and dynamic functional connectivity networks, discovering some functional connectivity features related to higher cognitive functions. Unfortunately, this method's later fusion is relatively simple, using only simple splicing operations and lacking proximity interactions between multi-view features. Subsequently, researchers proposed a static-dynamic convolutional neural network (SD-CNN), which increases the interaction of view features between static and dynamic paths through diffuse connections, improving classification accuracy by nearly 8% in epilepsy. Some researchers have designed a multi-connectivity representation learning network including structure-function and static-dynamic fusion modules to distinguish patients with major depressive disorder. Unfortunately, these methods only consider inter-class feature differences and ignore the high-level synergistic relationships between static and dynamic features of FBNs, such as inter-class and intra-class differences. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a brain network feature classification system and its training method based on dual collaborative learning. It utilizes deep learning frameworks such as convolutional neural networks, attention mechanisms, and contrastive learning to learn collaborative feature representations of high-level, calm, and dynamic brain networks and perform feature classification through a data-driven approach.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A brain network feature classification system based on dual collaborative learning, characterized in that it includes:

[0008] The brain network construction module obtains the average time series of each brain region based on the AAL template for each subject sample, uses the Pearson correlation coefficient to measure the functional connectivity strength between the time series of paired brain regions, and constructs the static functional brain network and dynamic functional brain network of the subject sample respectively.

[0009] The co-encoding module extracts edge-to-edge, edge-to-point, and point-to-graph hierarchical features from the static and dynamic functional brain networks, respectively. During the extraction of hierarchical features, the Transformer module is used to perform pruning and grafting operations to achieve co-learning of hierarchical features between the static and dynamic functional brain networks, and output static and dynamic co-encoding features.

[0010] The classification module uses a multilayer perceptron to classify the joint features obtained by concatenating static and dynamic collaborative features.

[0011] To optimize the above technical solution, the specific measures also include:

[0012] Furthermore, for each subject sample, the brain network construction module divides the brain of each subject into N brain regions based on the AAL template, and obtains the average time series of each brain region based on rs-fMRI data; the static functional brain network describes the structural topology information of the brain network, and the dynamic functional brain network captures the functional connectivity changes of the brain network through a time window.

[0013] Furthermore, the co-coding module uses a convolutional neural network as a feature extractor to sequentially extract edge-to-edge, edge-to-point, and point-to-graph hierarchical features from the static functional brain network and the dynamic functional brain network, respectively.

[0014] Furthermore, the co-coding module uses the Transformer module for pruning and grafting operations to achieve collaborative learning of hierarchical features between the static and dynamic functional brain networks, outputting static and dynamic co-coding features, specifically:

[0015] Edge-to-edge features of static and dynamic functional brain networks are extracted separately and denoted as static edge-to-edge features and dynamic edge-to-edge features, respectively. The static edge-to-edge features and dynamic edge-to-edge features are then aligned to the same feature dimension through downsampling.

[0016] The static edge-to-edge features and dynamic edge-to-edge features are input into the first Transformer module. The first Transformer module combines the scoring function to prune the static edge-to-edge features and dynamic edge-to-edge features with scores below a set value. The pruned parts of the static edge-to-edge features are then projected onto the corresponding dynamic edge-to-edge features to obtain static collaborative edge-to-edge features. The pruned parts of the dynamic edge-to-edge features are then projected onto the corresponding static edge-to-edge features to obtain dynamic collaborative edge-to-edge features.

[0017] The edge-to-point features of the static functional brain network are extracted from the static collaborative edge-to-edge features and the fusion features of the static edge-to-edge features, and are denoted as static edge-to-point features. The edge-to-point features of the dynamic functional brain network are extracted from the dynamic collaborative edge-to-edge features and the fusion features of the dynamic edge-to-edge features, and are denoted as dynamic edge-to-point features. The static edge-to-point features and the dynamic edge-to-point features are aligned to the same feature dimension through a downsampling operation.

[0018] The static edge-to-vertex features and dynamic edge-to-vertex features are input into the second Transformer module. The second Transformer module combines the scoring function to prune static edge-to-vertex features and dynamic edge-to-vertex features with scores below a set value. The pruned portion of the static edge-to-vertex features is then projected onto the corresponding dynamic edge-to-vertex features to obtain static collaborative edge-to-vertex features. The pruned portion of the dynamic edge-to-vertex features is then projected onto the corresponding static edge-to-vertex features to obtain dynamic collaborative edge-to-vertex features.

[0019] Point-to-graph features of static functional brain networks are extracted from static collaborative edge-to-point features and the fusion features of static edge-to-point features, and then obtained through batch standardization and activation functions to obtain static collaborative features; point-to-graph features of dynamic functional brain networks are extracted from dynamic collaborative edge-to-point features and the fusion features of dynamic edge-to-point features, and then obtained through batch standardization and activation functions to obtain dynamic collaborative features.

[0020] Furthermore, the multilayer perceptron used in the classification module includes three fully connected layers and one softmax layer.

[0021] Furthermore, all three fully connected layers use rectified linear units as activation functions and employ dropout of 0.12.

[0022] Furthermore, this invention also proposes a training method for a brain network feature classification system as described above, characterized by comprising:

[0023] The subject samples with known sample classification are input into the brain network construction module to construct static and dynamic functional brain networks for the subject samples, respectively.

[0024] For the static and dynamic collaborative features output by the co-coding module, comparative learning is performed between samples of the same class and between samples of different classes. Based on the results of the comparative learning, the parameters of the co-coding module are adjusted, and static and dynamic collaborative features that conform to the known sample classification are selected.

[0025] The static and dynamic collaborative features that match the known sample classification are concatenated and input into the classification module. Based on the feature classification results and the known sample classification, the parameters of the classification module are adjusted.

[0026] Furthermore, the comparative learning is performed between samples of the same category and between samples of different categories. Based on the results of the comparative learning, the parameters of the co-coding module are adjusted, specifically as follows:

[0027] The static and dynamic collaborative features output by the collaborative coding module are input into their respective multilayer perceptrons and standardized using the L2 norm.

[0028] Between samples of the same class and between samples of different classes, calculate the Euclidean distance between static co-features and the Euclidean distance between dynamic co-features, and combine the two as a measure of similarity;

[0029] Based on the similarity measurement results and known sample classifications, adjust the parameters of the co-coding module.

[0030] The beneficial effects of this invention are as follows: Based on the average time-series signal dependent on blood oxygenation level obtained from rs-fMRI data, this invention constructs two functional connectivity networks—static and dynamic—using Pearson correlation coefficient and sliding window techniques, respectively. Simultaneously, it proposes a dual cooperative learning network (DCLNet) that jointly learns the representations of static and dynamic functional brain networks through cooperative encoding and cooperative contrastive learning, considering high-level inter-class and intra-class feature relationships. The brain network feature classification system proposed in this invention not only significantly improves the performance of brain network feature classification but also provides new insights into the interactive dynamics of brain activity. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the principle of the brain network feature classification system based on dual collaborative learning proposed in this invention.

[0032] Figure 2 These are the top ten most discriminative brain regions selected in the ADHD vs. NC binary classification experiment according to embodiments of the present invention.

[0033] Figure 3 These are the top ten most discriminative brain regions selected in the AD vs. NC binary classification experiment according to embodiments of the present invention. Detailed Implementation

[0034] The invention will now be described in further detail with reference to the accompanying drawings.

[0035] like Figure 1 As shown, this invention proposes a brain network feature classification system based on dual collaborative learning, specifically a dual collaborative learning network (DCLNet), comprising the following modules:

[0036] The brain network construction module, for each subject sample, obtains the average time series of each brain region based on the Automated Anatomical Labeling (AAL) template, uses the Pearson correlation coefficient to measure the functional connectivity strength between the time series of paired brain regions, and constructs the static functional brain network (sFBN, sFBN) and dynamic functional brain network (dFBN) of the subject sample respectively.

[0037] The co-encoding module extracts edge-to-edge, edge-to-point, and point-to-graph hierarchical features from the static functional brain network and the dynamic functional brain network, respectively. During the extraction of hierarchical features, the Transformer module is used to perform pruning and grafting operations to achieve co-learning of hierarchical features between the sFBN and dFBN, and output static co-encoding features and dynamic co-encoding features.

[0038] The classification module uses a multilayer perceptron to classify the joint features obtained by concatenating static and dynamic collaborative features.

[0039] Next, the principles and functions of DCLNet will be explained in detail using the training method of the brain network feature classification system based on dual collaborative learning. The training method specifically includes the following steps:

[0040] S1: For each subject sample, based on the AAL template, the brain of each subject was divided into 116 brain regions, and the average time series of each brain region was obtained based on rs-fMRI data. Then, the Pearson correlation coefficient was used to measure the functional connectivity strength of paired brain regions, and static and dynamic functional brain networks were constructed accordingly. The methods for constructing static and dynamic functional brain networks are described in existing technologies, and therefore will not be elaborated here.

[0041] S2: For sFBN, a convolutional neural network is used as the feature extractor to sequentially obtain hierarchical feature representations of the FBN from edge to edge, edge to vertex, and vertex to graph (from local to global, low-order to high-order). For dFBN, similar to sFBN, the same convolutional operation is used for each sliding time window. In this way, the static path (i.e., sFBN) describes the static topology of functional connections, while the dynamic path (i.e., dFBN) captures the changing patterns of functional connections over time.

[0042] S3: A prune-graft transformer (PGT) is proposed to achieve collaborative learning between features. It uses a multi-head self-attention mechanism and pruning / grafting operations to dynamically adjust features between and within the static and dynamic views (i.e., sFBN and dFBN). The basic idea of ​​PGT is to dynamically prune inefficient feature units and replace them with projected aligned features from complementary views. In this way, much of the information within the original sFBN and dFBN is preserved, while the collaborative interaction between the two improves performance.

[0043] Taking the first-stage PGT as an example, this invention first aligns the edge-to-edge features extracted by the static and dynamic brain networks to the same feature dimension through downsampling. These features are then input into a Transformer for in-view feature updates. After the Transformer's feedforward module, this invention defines a scoring function for the output features to dynamically detect the feature vectors with the lowest scores in the bottom 20%, and projects them onto the corresponding view's features. Since the features of the two views are already spatially aligned, to improve the expressive power of the features, this invention uses a 1×1 convolution with non-linear activation to implement the projection operation. Finally, this invention fuses the static and dynamic collaborative edge-to-edge features with their respective original static and dynamic edge-to-edge features through convolution and broadcasting mechanisms. In this way, PGT retains the information within the original sFBN and dFBN while enhancing the collaborative effect between the sFBN and dFBN.

[0044] S4: To further learn high-level feature representations, a co-contrast loss is designed to make the semantic information of the features of the sFBN and dFBN of samples of the same class more similar. Specifically, a multilayer perceptron (non-linear projection head) is added to each path after the co-encoding module. Subsequently, the static and dynamic co-encoding features are normalized using the L2 norm for subsequent distance measurement. Specifically, Euclidean distance is used as a similarity measure, so that the Euclidean distance of the co-encoding features of the sFBN and dFBN of patients with the same brain disease is as close as possible, while the Euclidean distance of the co-encoding features of the sFBN and dFBN of patients with different brain diseases is as far apart as possible, achieving inter-class separability and intra-class compactness. The so-called high-level features are the static and dynamic co-encoding features that conform to the known sample classification.

[0045] S5: Output the model classification results using a joint classifier. Specifically, the high-level features between sFBN and dFBN obtained through co-contrast learning are simply concatenated and fused, and then connected to three fully connected layers and one softmax layer to achieve brain network feature classification. All three fully connected layers use rectified linear units (ReLU) as activation functions and employ dropout of 0.12.

[0046] In specific embodiments, this invention utilizes two publicly available datasets: the Alzheimer's Disease Neuroimaging Initiative (ADNI) and the Attention Deficit Hyperactivity Disorder (ADHD) ADHD-200 dataset. The ADNI dataset includes 179 participants: 48 normal controls (NC), 50 patients with early MCI (eMCI), 45 patients with late MCI (lMCI), and 31 AD patients, totaling 563 scans. Data acquisition specifications are as follows: image resolution 2.29-3.31 mm, slice thickness 3.31 mm, echo time 30 ms, and repetition time 2.2-3.1 s. The ADHD-200 dataset includes 216 participants: 98 normal controls and 118 ADHD patients, totaling 792 scans. The data acquisition specifications are as follows: the volumes of the first four echo plane imaging (EPI) images were removed to achieve signal equalization; the slice timing and head motion were corrected; the averaged planar echo images were co-registered to a 4×4×4 MNI template space; and the effects of white matter, cerebrospinal fluid, and head motion were removed.

[0047] In this embodiment, five-fold cross-validation is used to perform three sets of classification tasks to verify the effectiveness of the system. These include two binary classification tasks: ADHD vs. NC and AD vs. NC classification, and one multi-class classification task: AD vs. 1MCI vs. eMCI vs. NC classification. Specifically, for each classification task, all subjects are divided into five roughly equal subsets, and one subset is selected sequentially as the test set, while the remaining four subsets are used as the training set. Furthermore, during each cross-validation process, 20% of the training set data is selected as validation data to determine the optimal parameters of the model.

[0048] First, we compare the DCLNet method with the BrainNetCNN method. Specifically, the DCLNet method defines three convolutional filters: edge-to-edge, edge-to-point, and point-to-graph, to mine the topological features of the brain network structure. For the STNet method, an end-to-end spatiotemporal convolutional recurrent neural network is used. For the Wek-CNN method, a weight-correlated kernel-based convolutional neural network framework is constructed to learn hierarchical local-to-global, low-order-to-high-order feature representations. For the CRNN method, an end-to-end convolutional recurrent neural network is constructed. For the SD-CNN method, a static and dynamic convolutional neural network is used, with diffusion connections increasing the interaction of view features between static and dynamic channels.

[0049] To further evaluate the impact of pruning and co-contrast learning on the results in DCLNet, corresponding ablation experiments were conducted on DCLNet. Specifically, the co-contrast learning module (denoted as DCLNet-C), the pruning and grafting Transformer module (denoted as DCLNet-PGT), and both were removed (denoted as DCLNet-PGT-C) were removed respectively. Tables 1 and 2 report the quantitative results obtained by the nine methods in binary and multi-class classification tasks, respectively.

[0050] Table 1. Performance of nine different methods in ADHD vs. NC classification and AD vs. NC classification.

[0051]

[0052]

[0053] Table 2 shows the performance of nine different methods on the AD vs. 1MCI vs. eMCI vs. NC classification tasks.

[0054]

[0055] Tables 1 and 2 summarize the performance of the nine methods for binary and multi-class classification tasks, respectively. As can be seen from Tables 1 and 2, for all classification tasks (the best results are highlighted in bold), DCLNet and its variants achieved top-1 classification accuracy. Figure 2 This shows the abnormal brain regions detected by DCLNet in the ADHD identification task, such as the precentral gyrus, superior frontal gyrus (dorsal), and orbitofrontal cortex (middle). These brain regions are closely associated with typical ADHD symptoms, such as inattention and difficulty planning. In the AD vs. NC classification task ( Figure 3DCLNet identified several key brain regions involved in important neural structures such as the amygdala and hippocampus. Notably, the detected abnormal brain regions were primarily located in the temporal lobe. Generally, the temporal lobe is one of the earliest and most severely damaged brain regions in Alzheimer's disease, and changes in its structure and function can reflect the development and severity of Alzheimer's. Therefore, DCLNet effectively identifies abnormal brain regions associated with ADHD and AD, providing valuable insights for the field of neuroimaging.

[0056] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A brain network feature classification system based on dual collaborative learning, characterized in that, include: The brain network construction module obtains the average time series of each brain region based on the AAL template for each subject sample, uses the Pearson correlation coefficient to measure the functional connectivity strength between the time series of paired brain regions, and constructs the static functional brain network and dynamic functional brain network of the subject sample respectively. The co-encoding module extracts hierarchical features from both the static and dynamic functional brain networks, sequentially at the edge-to-edge, edge-to-vertex, and vertex-to-graph levels. During feature extraction, the Transformer module performs pruning and grafting operations to achieve co-learning of these hierarchical features between the static and dynamic functional brain networks, outputting static and dynamic co-encoding features. Edge-to-edge features of static and dynamic functional brain networks are extracted separately and denoted as static edge-to-edge features and dynamic edge-to-edge features, respectively. The static edge-to-edge features and dynamic edge-to-edge features are then aligned to the same feature dimension through downsampling. The static edge-to-edge features and dynamic edge-to-edge features are input into the first Transformer module. The first Transformer module combines the scoring function to prune the static edge-to-edge features and dynamic edge-to-edge features with scores below a set value. The pruned parts of the static edge-to-edge features are then projected onto the corresponding dynamic edge-to-edge features to obtain static collaborative edge-to-edge features. The pruned parts of the dynamic edge-to-edge features are then projected onto the corresponding static edge-to-edge features to obtain dynamic collaborative edge-to-edge features. The edge-to-point features of the static functional brain network are extracted from the static collaborative edge-to-edge features and the fusion features of the static edge-to-edge features, and are denoted as static edge-to-point features. The edge-to-point features of the dynamic functional brain network are extracted from the dynamic collaborative edge-to-edge features and the fusion features of the dynamic edge-to-edge features, and are denoted as dynamic edge-to-point features. The static edge-to-point features and the dynamic edge-to-point features are aligned to the same feature dimension through a downsampling operation. The static edge-to-vertex features and dynamic edge-to-vertex features are input into the second Transformer module. The second Transformer module combines the scoring function to prune static edge-to-vertex features and dynamic edge-to-vertex features with scores below a set value. The pruned portion of the static edge-to-vertex features is then projected onto the corresponding dynamic edge-to-vertex features to obtain static collaborative edge-to-vertex features. The pruned portion of the dynamic edge-to-vertex features is then projected onto the corresponding static edge-to-vertex features to obtain dynamic collaborative edge-to-vertex features. Point-to-graph features of static functional brain networks are extracted from static collaborative edge-to-point features and the fusion features of static edge-to-point features, and then obtained through batch standardization and activation functions to obtain static collaborative features; point-to-graph features of dynamic functional brain networks are extracted from dynamic collaborative edge-to-point features and the fusion features of dynamic edge-to-point features, and then obtained through batch standardization and activation functions to obtain dynamic collaborative features. The classification module uses a multilayer perceptron to classify the joint features obtained by concatenating static and dynamic collaborative features.

2. The brain network feature classification system based on dual collaborative learning as described in claim 1, characterized in that: For each subject sample, the brain network construction module divides the brain of each subject into N brain regions based on the AAL template, and obtains the average time series of each brain region based on rs-fMRI data; the static functional brain network describes the structural topology information of the brain network, and the dynamic functional brain network captures the functional connectivity changes of the brain network through time windows.

3. The brain network feature classification system based on dual collaborative learning as described in claim 1, characterized in that: The co-coding module uses a convolutional neural network as a feature extractor to extract layered features from static functional brain networks and dynamic functional brain networks, respectively, for edge-to-edge, edge-to-point, and point-to-graph.

4. The brain network feature classification system based on dual collaborative learning as described in claim 1, characterized in that: The classification module uses a multilayer perceptron consisting of three fully connected layers and one softmax layer.

5. The brain network feature classification system based on dual collaborative learning as described in claim 4, characterized in that: All three fully connected layers use rectified linear units as activation functions and employ dropout of 0.

12.

6. A training method for a brain network feature classification system as described in any one of claims 1-5, characterized in that, include: The subject samples with known sample classification are input into the brain network construction module to construct static and dynamic functional brain networks for the subject samples, respectively. For the static and dynamic collaborative features output by the co-coding module, comparative learning is performed between samples of the same class and between samples of different classes. Based on the results of the comparative learning, the parameters of the co-coding module are adjusted, and static and dynamic collaborative features that conform to the known sample classification are selected. The static and dynamic collaborative features that match the known sample classification are concatenated and input into the classification module. Based on the feature classification results and the known sample classification, the parameters of the classification module are adjusted.

7. The training method for the brain network feature classification system as described in claim 6, characterized in that: The process involves comparative learning between samples of the same category and between samples of different categories. Based on the results of this comparative learning, the parameters of the co-coding module are adjusted. Specifically: The static and dynamic collaborative features output by the collaborative coding module are input into their respective multilayer perceptrons and standardized using the L2 norm. Between samples of the same class and between samples of different classes, calculate the Euclidean distance between static co-features and the Euclidean distance between dynamic co-features, and combine the two as a measure of similarity; Based on the similarity measurement results and known sample classifications, adjust the parameters of the co-coding module.

Images