Brain image hypergraph processing system and method fusing brain function imaging and clinical data

Abstract

The invention discloses a brain image hypergraph processing system fusing brain function imaging and clinical data. Comprising a global static brain function connection super-network module for constructing a global static brain function connection super-network, a local dynamic brain function connection super-network module for constructing a local dynamic brain function connection super-network, and a super-graph neural network classification model. The hypergraph neural network classification model comprises a feature learning device, a first feature fusion device and a feature classifier; the feature learning device comprises a dynamic branch feature learning device, a static branch feature learning device and an individual clinical feature learning module; performing feature extraction on local dynamic and global static brain function connection super networks by dynamic and static branch feature learners; a first feature fusion device fuses features extracted by the dynamic and static branch feature learning devices and the individual clinical feature learning module, and a feature classifier classifies the features fused by the first feature fusion device. According to the invention, the recognition accuracy on the fMRI data set is improved.

Description

Technical Field

[0001] The present invention relates to a method for processing brain functional imaging, and particularly to a brain image hypergraph processing system and method for integrating brain functional imaging and clinical data. Background Art

[0002] Currently, computer-aided diagnosis (CAD) has attracted much attention in the research field of major depressive disorder (MDD). At home and abroad, the diagnosis of depression mainly relies on psychological scales to evaluate major depressive disorder. However, these scales are easily interfered by factors such as age, education level, and learning effects, and their application effects in clinical practice are limited. Therefore, in clinical practice, patients often need to show significant depressive symptoms before they can be diagnosed. And early and accurate diagnosis of this disease is of great significance for timely treatment and reducing the risk of patients. Therefore, accurately evaluating the disease status with the help of a CAD system is crucial for the prognosis and treatment of patients.

[0003] Resting-state functional Magnetic Resonance Imaging (Rs-fMRI) is an advanced magnetic resonance technology for studying the central nervous system, which has the advantages of high sensitivity, fast imaging speed, and high signal-to-noise ratio, and plays a key role in exploring early brain functional changes in diseases. In recent years, the research on depression based on Rs-fMRI has gradually received extensive attention, which provides new hope for the early diagnosis and pathophysiological mechanism research of major depressive disorder.

[0004] Combined with the complex network theory of graph theory, research shows that the brain functional network constructed using functional magnetic resonance imaging data has many important topological properties, and major depressive disorder is closely related to the abnormal changes in the topological structure of the brain functional network. These studies not only provide a new perspective for revealing the pathological mechanism of major depressive disorder, but also provide brain network imaging markers for the early diagnosis and treatment effect evaluation of the disease. Traditional functional connection networks are usually based on the pairwise correlation between brain regions, mainly reflecting the relationship between paired brain regions, and ignoring the complex interactions between multiple brain regions. However, from a neurological perspective, a brain region usually interacts with multiple brain regions, and the loss of this important information may have a negative impact on the accuracy of disease diagnosis.

[0005] Previously, a study proposed constructing a brain functional connectivity hypernetwork through a hypergraph method, which can more effectively represent the interaction relationships between multiple brain regions. At the same time, learning the characteristics of the brain functional connectivity hypernetwork is also an important research direction. Only by effectively extracting network features can accurate disease diagnosis be achieved. In addition, fMRI datasets usually contain a large amount of noise, mainly from two sources: one is the noise information contained in the collected data affected by factors such as the environment and the subject's state; the other is that since large-scale fMRI data usually comes from multiple sites, the instrument characteristics, acquisition methods, etc. of different sites lead to inevitable site differences in cross-site data. The noise in these data may have a serious impact on the recognition effect of computer models. How to effectively remove this noise plays a key role in improving the diagnostic performance.

[0006] Traditional methods mostly focus on feature mining for fMRI data. In addition to using brain fMRI information, subjects usually have a series of individual characteristics, such as age, gender, education, medical conditions, etc. These clinical factors may also contribute to disease recognition to a certain extent. Exploring the deep features contained therein through a neural network model and integrating them with fMRI features is a very worthy direction. Summary of the Invention

[0007] The present invention provides a brain imaging hypergraph processing system and method that integrates brain functional imaging and clinical data to solve the technical problems existing in the prior art.

[0008] The technical solution adopted by the present invention to solve the technical problems existing in the prior art is as follows:

[0009] A brain imaging hypergraph processing system that integrates brain functional imaging and clinical data includes a global static brain functional connectivity hypernetwork module, a local dynamic brain functional connectivity hypernetwork module, and a hypergraph neural network classification model; randomly select a subject from the subject set, and call this subject subject U;

[0010] The global static brain functional connectivity hypernetwork module constructs a functional connection vector between each brain region and other brain regions based on the correlation of subject U's brain imaging data in the order of brain regions, generates a global static correlation coefficient matrix; then based on the global static correlation coefficient matrix, generates a global static functional connectivity hypernetwork adjacency matrix; and further generates the global static brain functional connectivity hypernetwork of this subject;

[0011] The local dynamic brain functional connectivity hypernetwork module, based on the sliding window method, divides the brain imaging time series data of each brain region of subject U into multiple segments, calculates the Pearson correlation coefficients of these segments, generates a series of local dynamic correlation coefficient matrices of brain regions, and then generates a series of local dynamic brain functional connectivity hypernetworks of this subject from each local dynamic correlation coefficient matrix of each brain region;

[0012] The hypergraph neural network classification model includes a feature learner, a first feature fusion unit, and a feature classifier; the feature learner includes a dynamic branch feature learner, a static branch feature learner, and an individual clinical feature learning module;

[0013] The dynamic branch feature learner is used to extract features from a series of local dynamic functional connection hypernetworks of the subject U;

[0014] The static branch feature learner is used to extract features from the global static brain functional connection hypernetwork of the subject U;

[0015] The individual clinical feature learning module is used to extract features from the clinical brain imaging data of the subject U or patients with the same symptoms as the subject U;

[0016] The first feature fusion unit is used to fuse the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module;

[0017] The feature classifier is used to classify the features fused by the first feature fusion unit.

[0018] Furthermore, the hypergraph neural network classification model further includes a site classifier; the site classifier is used to predict the detection site to which the subject U belongs through the features fused by the first feature fusion unit; it calculates the cross-entropy loss with the true site as the target label; and uses the backpropagation algorithm to update the model weights of the site classifier to improve the ability of the site classifier to distinguish interference factors caused by site differences from the features extracted by the feature learner.

[0019] Furthermore, the static branch feature learner includes a first hypergraph convolution module, a first information enhancement module, and a first spatial attention module connected in sequence; the dynamic branch feature learner includes a second hypergraph convolution module, a second spatial attention module, a GRU module, a temporal attention module, a second information enhancement module, and a third spatial attention module;

[0020] The first and second hypergraph convolution modules are used to update the features of the nodes by simulating the propagation and aggregation of information in the hyperedges;

[0021] The first and second information enhancement modules are used to reduce data noise and strengthen feature representation;

[0022] The first and second spatial attention modules are used to enhance the sensitivity to specific spatial regions in the input data;

[0023] The GRU module is used to extract the features of sequential data;

[0024] The temporal attention module is used to enhance the sensitivity to specific sequential regions in the input time series data;

[0025] The output end of the second hypergraph convolution module is respectively connected to the input ends of the second spatial attention module and the GRU module; the output ends of the second spatial attention module and the GRU module are connected to the input end of the temporal attention module; the output end of the temporal attention module is connected to the input end of the second information enhancement module; the output end of the second information enhancement module is connected to the input end of the third spatial attention module; the output ends of the second and third spatial attention modules are connected to the input end of the first feature fusion device.

[0026] Furthermore, the dynamic branch feature learner further includes a second feature fusion device; the second feature fusion device is used to fuse the features output by the first and second spatial attention modules; the input end of the second feature fusion device is connected to the output ends of the first and second spatial attention modules; the output end of the second feature fusion device is connected to the input end of the temporal attention module.

[0027] The present invention also provides a method for processing brain image hypergraphs by fusing brain functional imaging and clinical data using the above-mentioned brain image hypergraph processing system for fusing brain functional imaging and clinical data. A large amount of resting-state fMRI data of subjects across sites is collected and preprocessed. The brain regions marked by the AAL template are used as regions of interest, and the average time series of each brain region is obtained. By the global static brain functional connectivity hypernetwork module, based on the brain image data of each brain region of each subject, a regression method is used to construct a global static brain functional connectivity hypernetwork. By the local dynamic brain functional connectivity hypernetwork module, using the sliding window method and the Pearson correlation coefficient method, based on the time series of each brain region of each subject, a series of local dynamic brain functional connectivity hypernetworks of the subject are constructed. By the individual clinical feature learning module, feature extraction is performed on the clinical brain image data of the subject or patients with the same symptoms as the subject. By the dynamic branch feature learner, feature extraction is performed on a series of local dynamic functional connectivity hypernetworks of the subject. By the static branch feature learner, feature extraction is performed on the global static brain functional connectivity hypernetwork of the subject. By the first feature fusion device, the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module are fused, and the features fused by the first feature fusion device are classified by the feature classifier.

[0028] Furthermore, the method includes the following method steps:

[0029] Calculate the LASSO regression correlation coefficient of the fMRI time series between brain regions of each subject to obtain the correlation between brain regions, and construct a functional connection vector between each brain region and other brain regions based on the correlation in the order of brain regions to obtain a global static correlation coefficient matrix where N is the number of brain regions and M is the number of hyperedges;

[0030] Based on the global static correlation coefficient matrix H of each subject, set the matrix elements with values less than t to 0. After filtering H, the global static functional connectivity hypernetwork adjacency matrix of the subject is obtained. Each column vector is a hyperedge. Let i be the brain region number, and the central node of the i-th column vector is the i-th brain region, and other nodes are the neighbor nodes in this hyperedge. The correlation coefficient between it and the central node is the correlation between brain regions. All hyperedges form a global static brain functional connectivity hypernetwork G;

[0031] Based on the sliding window method, according to a fixed time length w and time interval s, divide the fMRI time series of each brain region with a duration of T into segments, calculate the Pearson correlation coefficients of the fMRI time series segments between these brain regions, and obtain a series of local dynamic correlation coefficient matrices P D , P D =(P1,...,P n ), r is the serial number of the local dynamic correlation coefficient matrix; r = 1, 2,..., n; n is the number of local dynamic correlation coefficient matrices;

[0032] The r-th local dynamic correlation coefficient matrix of the i-th brain region is P ri . According to P ri , calculate the distance between the i-th brain region and other brain regions, and take the first k brain regions with the smallest distance from the i-th brain region as the neighbor nodes of the i-th brain region to construct a hyperedge; this hyperedge takes the i-th brain region as the central node, and other nodes as the neighbor nodes in the hyperedge; taking all brain regions as the central nodes, repeat this step to obtain a series of local dynamic brain functional connectivity hypernetworks G D =(G1,...,G q ), v is the serial number of the local dynamic brain functional connectivity hypernetwork; v = 1, 2,..., q; q is the number of local dynamic brain functional connectivity hypernetworks;

[0033] According to the order of each brain region in the brain region division, encode it in the form of a one-hot vector, and encode the position order of the brain region in the sub-functional brain network in the form of a one-hot vector. Concatenate the two vectors to obtain the hypernetwork node feature, that is, the node feature corresponding to the i-th brain region is where S is the number of sub-functional brain networks, and in this way, the position information of different dimensions of the brain region is correspondingly incorporated into the hypergraph features of the local dynamic brain functional connectivity hypernetwork and the global static brain functional connectivity hypernetwork.

[0034] Furthermore, use the following method steps to train the hypergraph neural network classification model:

[0035] Step A1, collect the resting-state fMRI data of the subjects for preprocessing, and add individual data such as education, age, gender, and medical conditions corresponding to the subjects; randomly divide the preprocessed fMRI data into a training set and a test set, normalize the feature data in the training set to a distribution with a mean of 0 and a variance of 1, and save the mean and variance of the training set at the same time;

[0036] Step A2, take different values for w, s, t, and k, and process the training set and the test set by the global static brain functional connectivity hypernetwork module to obtain the global static brain functional connectivity hypernetwork training set and test set correspondingly; process the training set and the test set by the local dynamic brain functional connectivity hypernetwork module to obtain the local dynamic brain functional connectivity hypernetwork training set and test set correspondingly;

[0037] The training set of the global static brain functional connectivity hypernetwork and the training set of the local dynamic brain functional connectivity hypernetwork are used to train the hypergraph neural network classification model, and the test set of the global static brain functional connectivity hypernetwork and the test set of the local dynamic brain functional connectivity hypernetwork are used to test the classification effect of the hypergraph neural network classification model;

[0038] Step A3, corresponding to the same subject, send the training set of the global static brain functional connectivity hypernetwork into the static branch feature learner, and send the training set of the local dynamic brain functional connectivity hypernetwork into the dynamic branch feature learner; use the first feature fusion device to splice the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module for the same subject, and use the feature classifier to predict the brain image category of the subject, and calculate the cross-entropy loss with its true brain image category as the target label y e is the true brain image label of the e-th sample, p e is the probability that the e-th sample is predicted as a certain brain image category, and e is the training sample number; E is the number of training samples; use the backpropagation algorithm to update the model weights of the feature learner and the feature classifier to improve the ability of the feature learner to learn disease-related brain features.

[0039] Furthermore, the hypergraph neural network classification model is also equipped with a site classifier; the site classifier is used to predict the detection site to which the subject belongs through the features fused by the first feature fusion device; it calculates the cross-entropy loss with the true site as the target label; use the backpropagation algorithm to update the model weight of the site classifier to improve the ability of the site classifier to distinguish the interference factors caused by site differences from the features extracted by the feature learner.

[0040] Furthermore, the steps for training the hypergraph neural network classification model also include the following:

[0041] Step A4: Concatenate the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module for the same subject, and use the site classifier to predict the detection site to which the subject belongs. Calculate the cross-entropy loss L with its true site as the target label. d , F is the number of sites, y e,f is the sign function, which is 1 when the true site label of the e-th sample is f, otherwise 0, and p e,f is the probability that the predicted site label of the e-th sample is f. Use the backpropagation algorithm to update the model weights of the site classifier to enhance the ability of the site classifier to distinguish interference factors caused by site differences from the features extracted by the feature learner.

[0042] Step A5: Concatenate the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module for the same subject, and use the site classifier to predict the detection site to which the subject belongs. Calculate the confusion loss with its true site as the target label. Use the backpropagation algorithm to update the model weights of the feature learner to reduce the interference factors caused by site differences in the features extracted by the feature learner.

[0043] Step A6: Repeat Step A3 to Step A5 until the model converges. The overall loss function is L = L p + αL d + βL conf .

[0044] Furthermore, use the grid search method to obtain the optimal values of the following parameters: w, s, t, k, α, β, and different hidden layer dimension values of GRU.

[0045] The advantages and positive effects of the present invention are:

[0046] The present invention uses LASSO regression, Pearson correlation coefficient, and sliding window method to construct a global static and local dynamic functional connection hypernetwork to characterize brain spatial and temporal features; integrates the individual clinical features of the subject to further enrich multimodal information; constructs a hypergraph neural network classification model and uses it to perform feature integration, enhancement, learning, and classification based on the constructed hypergraph and individual clinical features. The model consists of a hypergraph neural network feature learner, a feature classifier, and a site classifier, and its training process completes steps such as feature extraction, feature enhancement, and interference factor reduction based on mechanisms such as hypergraph convolution, spatio-temporal attention learning, information enhancement, and site difference learning; uses the grid search method to adjust and optimize the parameters of the functional connection hypernetwork and the hypergraph neural network classification model, and uses the optimized functional connection hypernetwork and classification model to classify brain imaging hypergraphs.

[0047] The present invention constructs a brain functional connectivity hypernetwork through functional connectivity information, which can more effectively characterize the interaction relationships between multiple brain regions while retaining the key information of functional connectivity and excluding interference factors. Therefore, the method proposed by the present invention can help us understand the mechanisms of brain operation and brain diseases more deeply.

[0048] The present invention expands the data source, uses a neural network to mine individual clinical characteristics and integrates them with brain fMRI characteristics. Compared with feature learning based on single fMRI data, it can more fully characterize brain characteristics and disease-related information.

[0049] The present invention adds a site difference interference elimination module and integrates it into the model training process to reduce the interference of data acquisition site differences on feature learning. Compared with the brain network features constructed directly based on cross-site data sets in the traditional way, it can more accurately characterize the complex functional interactions between brain regions.

[0050] Compared with other computer-aided detection methods based on hypergraphs, the present invention reduces the interference of data acquisition site differences on feature learning to a certain extent, further enriches the source of subject features, adds individual clinical information, integrates it with brain fMRI characteristics, and significantly improves the recognition accuracy and robustness of the hypergraph neural network classification model on the fMRI data set. Brief Description of the Drawings

[0051] Figure 1 is a flowchart of a brain imaging hypergraph processing method that integrates brain functional imaging and clinical data of the present invention.

[0052] Figure 2 is a flowchart of hyperedge construction for the global static brain functional connectivity hypernetwork and a series of local dynamic brain functional connectivity hypernetworks.

[0053] Figure 3 is a working principle diagram of the hypergraph neural network classification model.

[0054] In the figure:

[0055] H: Global static correlation coefficient matrix;

[0056] P1~P n : The 1st to the nth local dynamic correlation coefficient matrices;

[0057] h 11 ~h MN : Elements of the global static correlation coefficient matrix H;

[0058] P 11 ~P MN : Elements of the local dynamic correlation coefficient matrix;

[0059] GRU: Gated Recurrent Unit, a deep learning recurrent neural network for feature learning of sequence data;

[0060] MLP: Multilayer Perceptron, that is, a multi - layer fully connected neural network, which can perform non - linear mapping on input data;

[0061] M: A temporal attention weight vector representing the influence degree of different local dynamic hypergraphs on the global static hypergraph, reflecting the importance of brain features at different time points;

[0062] H s : The deep - layer features of the global static hypergraph learned by the hypergraph convolution module;

[0063] H ′ s : The deep - layer features of the global static hypergraph enhanced by the information enhancement module;

[0064] Z s : The spatial attention weight of the global static hypergraph;

[0065] Z d : The spatial attention weight of the local dynamic hypergraph;

[0066] H d (1), H d (2),..., H d (n): The deep - layer features of the 1st to the nth local dynamic hypergraphs learned by the hypergraph convolution module;

[0067] The temporal features of the 1st to the nth local dynamic hypergraphs learned by the GRU module;

[0068] The deep - layer features of the local dynamic hypergraphs processed by the temporal attention module;

[0069] H ′ d : The deep - layer features of the local dynamic hypergraphs enhanced by the information enhancement module;

[0070] H g : The hypergraph features weighted by the spatial attention module;

[0071] The final hypergraph features obtained by concatenating the final features of the global static hypergraph and the final features of the local dynamic hypergraph. Detailed implementation manners

[0072] The present invention will be described in detail below with reference to the drawings and in conjunction with embodiments. It should be understood that the preferred embodiments described herein are only for explaining and illustrating the present invention, and are not used to limit the present invention.

[0073] The English words and abbreviations in the following terms are explained in Chinese as follows:

[0074] fMRI: Functional Magnetic Resonance Imaging.

[0075] Rs-fMRI: Resting-State Functional Magnetic Resonance Imaging.

[0076] AAL: Automated Anatomical Labeling atlas.

[0077] DPARSF software: A graph theory network analysis toolbox for brain imaging connectomics.

[0078] HGCN: Hypergraph Convolutional Network, a deep learning convolutional module for feature learning on hypergraphs.

[0079] GRU: Gated Recurrent Unit, a deep learning recurrent neural network for feature learning on sequential data.

[0080] LASSO regression: A linear regression method using L1 regularization. Using L1 regularization will make the weights of some learned features zero, thus achieving the purpose of sparsification and feature selection.

[0081] SPM: Widely used open-source fMRI processing software.

[0082] REST-meta-MDD dataset: A publicly available MDD dataset.

[0083] MLP: Multi-Layer Perceptron, that is, a multi-layer fully connected neural network, which can perform non-linear mapping on input data.

[0084] The terms "connected" and "connection" used in the present invention should be understood in a broad sense, referring to electrical connection or signal transmission; for those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0085] Please refer to Figures 1 to 3 , a brain imaging hypergraph processing system that fuses brain functional imaging and clinical data, including a global static brain functional connectivity hypernetwork module, a local dynamic brain functional connectivity hypernetwork module, and a hypergraph neural network classification model; randomly select a subject from the subject set, and call this subject subject U.

[0086] The global static brain functional connectivity hypernetwork module constructs a functional connection vector between each brain region and other brain regions based on correlation according to the brain region order of the brain image data of subject U, and generates a global static correlation coefficient matrix; then based on the global static correlation coefficient matrix, it generates a global static functional connectivity hypernetwork adjacency matrix; and further generates the global static brain functional connectivity hypernetwork of this subject.

[0087] A local dynamic brain functional connectivity hypernetwork module, which, based on the sliding window method, divides the brain imaging time series data of each brain region of subject U into multiple segments, calculates the Pearson correlation coefficients of these segments, generates a series of local dynamic correlation coefficient matrices of brain regions, and then generates a series of local dynamic brain functional connectivity hypernetworks of the subject from each local dynamic correlation coefficient matrix of each brain region.

[0088] The hypergraph neural network classification model includes a feature learner, a first feature fusion unit, and a feature classifier; the feature learner includes a dynamic branch feature learner, a static branch feature learner, and an individual clinical feature learning module.

[0089] The dynamic branch feature learner is used to extract features from a series of local dynamic functional connectivity hypernetworks of subject U.

[0090] The static branch feature learner is used to extract features from the global static brain functional connectivity hypernetwork of subject U.

[0091] The individual clinical feature learning module is used to extract features from the clinical brain imaging data of subject U or patients with the same symptoms as subject U.

[0092] The first feature fusion unit is used to fuse the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module.

[0093] The feature classifier is used to classify the features fused by the first feature fusion unit.

[0094] Preferably, the hypergraph neural network classification model may further include a site classifier; the site classifier can be used to predict the detection site to which subject U belongs through the features fused by the first feature fusion unit; it calculates the cross-entropy loss with the true site as the target label; and uses the backpropagation algorithm to update the model weights of the site classifier to enhance the ability of the site classifier to distinguish interference factors caused by site differences from the features extracted by the feature learner.

[0095] Preferably, the static branch feature learner may include a first hypergraph convolution module, a first information enhancement module, and a first spatial attention module connected in sequence; the dynamic branch feature learner may include a second hypergraph convolution module, a second spatial attention module, a GRU module, a temporal attention module, a second information enhancement module, and a third spatial attention module;

[0096] The first and second hypergraph convolution modules can be used to update the features of nodes by simulating the propagation and aggregation of information in hyperedges.

[0097] The first and second information enhancement modules can be used to reduce data noise and strengthen feature representation.

[0098] The first and second spatial attention modules can be used to enhance the sensitivity to specific spatial regions in the input data.

[0099] The GRU module can be used to extract the features of sequential data.

[0100] The temporal attention module can be used to enhance the sensitivity to specific sequential regions in the input time series data.

[0101] Among them, the output end of the second hypergraph convolution module can be respectively connected to the input ends of the second spatial attention module and the GRU module; the output ends of the second spatial attention module and the GRU module are connected to the input end of the temporal attention module; the output end of the temporal attention module is connected to the input end of the second information enhancement module; the output end of the second information enhancement module is connected to the input end of the third spatial attention module; the output ends of the second and third spatial attention modules are connected to the input end of the first feature fusion device. The first and second hypergraph convolution modules adopt HGCN.

[0102] Preferably, the dynamic branch feature learner may further include a second feature fusion device; the second feature fusion device can be used to fuse the features output by the first and second spatial attention modules; the input end of the second feature fusion device is connected to the output ends of the first and second spatial attention modules; the output end of the second feature fusion device is connected to the input end of the temporal attention module.

[0103] The training results of the corresponding spatial attention module and the corresponding temporal attention module parameters in the dynamic branch feature learner and the static branch feature learner can be used to distinguish the importance of different brain regions and different time periods for pathological features, and find effective biomarkers for major depressive disorder. The higher the weight of the attention module obtained, the more important the brain region is in the disease activity, and it may be an important biomarker, and its lesion may be one of the pathogenic factors.

[0104] The present invention also provides an embodiment of a method for processing brain image hypergraphs that integrates brain functional imaging and clinical data using the above-mentioned brain image hypergraph processing system that integrates brain functional imaging and clinical data. A large amount of resting-state fMRI data of subjects across multiple sites is collected and preprocessed. The brain regions marked using the AAL template are used as regions of interest, and the average time series of each brain region is obtained. By the global static brain functional connectivity hypernetwork module, based on the brain image data of each brain region of each subject, a regression method is used to construct a global static brain functional connectivity hypernetwork. By the local dynamic brain functional connectivity hypernetwork module, using the sliding window method and the Pearson correlation coefficient method, based on the time series of each brain region of each subject, a series of local dynamic brain functional connectivity hypernetworks of the subject are constructed. By the individual clinical feature learning module, feature extraction is performed on the clinical brain image data of the subject or patients with the same symptoms as the subject. By the dynamic branch feature learner, feature extraction is performed on a series of local dynamic functional connectivity hypernetworks of the subject. By the static branch feature learner, feature extraction is performed on the global static brain functional connectivity hypernetwork of the subject. By the first feature fusion device, the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module are fused, and the features fused by the first feature fusion device are classified by the feature classifier.

[0105] Preferably, the method may include the following method steps:

[0106] Calculate the LASSO regression correlation coefficients of the fMRI time series between brain regions of each subject to obtain the correlations between brain regions, and construct a functional connection vector between each brain region and other brain regions based on the correlations in the order of brain regions to obtain a global static correlation coefficient matrix where N is the number of brain regions and M is the number of hyperedges.

[0107] Based on the global static correlation coefficient matrix H of each subject, set the matrix element values less than t to 0. After H is filtered, a global static functional connectivity hypernetwork adjacency matrix of the subject is obtained, where each column vector is a hyperedge. Let i be the brain region serial number, and let the central node of the i-th column vector be the i-th brain region, and other nodes be the neighbor nodes in this hyperedge. The correlation coefficient between it and the central node is the correlation between brain regions. All hyperedges form a global static brain functional connectivity hypernetwork G.

[0108] Based on the sliding window method, according to a fixed time length w and time interval s, the fMRI time series of each brain region with a duration of T is divided into segments, and the Pearson correlation coefficients of these fMRI time series segments between brain regions are calculated to obtain a series of local dynamic correlation coefficient matrices P D , P D =(P1,...,P n ), r is the serial number of the local dynamic correlation coefficient matrix; r = 1, 2, …, n; n is the number of local dynamic correlation coefficient matrices.

[0109] The r-th local dynamic correlation coefficient matrix of the i-th brain region is P ri , according to P ri Calculate the distance between the i-th brain region and other brain regions, and take the top k brain regions with the smallest distance from the i-th brain region as the neighbor nodes of the i-th brain region to construct a hyperedge; this hyperedge takes the i-th brain region as the central node, and other nodes as the neighbor nodes in the hyperedge; repeat this step with all brain regions as the central nodes to obtain a series of local dynamic brain functional connection hypernetworks G D =(G1,...,G q ), v is the serial number of the local dynamic brain functional connection hypernetwork; v = 1, 2, …, q; q is the number of local dynamic brain functional connection hypernetworks.

[0110] The i-th brain region is the i-th brain region after the brain regions are divided by the AAL brain atlas. Each brain region has a set of correlation coefficient matrices obtained by the sliding window method and the Pearson correlation coefficient. The r-th correlation coefficient matrix is the r-th in this set of correlation coefficient matrices.

[0111] According to the order of each brain region in the brain region division, encode it in the form of a one-hot vector, and encode the position order of the brain region in the sub-functional brain network in the form of a one-hot vector, and splice the two vectors to obtain the hypernetwork node feature, that is, the node feature corresponding to the i-th brain region is where S is the number of sub-functional brain networks, and in this way, the position information of different dimensions of the brain region is correspondingly incorporated into the hypergraph features of the local dynamic brain functional connection hypernetwork and the global static brain functional connection hypernetwork.

[0112] The node feature is a part of the hypergraph feature, and the position information of different perspectives of the node can be used as the node feature.

[0113] The sub-functional brain network is the functional partition network of the brain and belongs to knowledge. After modeling the brain functional state as a hypergraph, each node in the hypergraph has its own node feature. Encode the position of the node in the AAL atlas and the position order in the known brain functional partition in the form of one-hot vectors respectively and then splice them to form the node feature.

[0114] Preferably, the following method steps can be used to train the hypergraph neural network classification model:

[0115] Step A1, collect the resting-state fMRI data of the subjects for preprocessing, and add individual data such as education, age, gender, and medical conditions corresponding to the subjects; randomly divide the preprocessed fMRI data into a training set and a test set, normalize the feature data in the training set to a distribution with a mean of 0 and a variance of 1, and at the same time save the mean and variance of the training set.

[0116] The DPARSF software can be used to preprocess the collected resting-state fMRI data. The preprocessing steps can be as follows: First, remove the first 5-10 time points, and then perform temporal slice timing correction, head motion correction, registration of functional images, removal of linear drift, band-pass filtering, and regression of covariates in sequence.

[0117] Use the five-fold cross-validation method for model training. Suppose there are L samples, randomly select L / 5 of them as test samples, and the other 4L / 5 samples as training samples to obtain L / 5 classification test results; use the average value of these L / 5 classification test results to evaluate the model performance.

[0118] Step A2, take different values for w, s, t, and k, and process the training set and the test set by the global static brain functional connectivity hypernetwork module to obtain the global static brain functional connectivity hypernetwork training set and test set; process the training set and the test set by the local dynamic brain functional connectivity hypernetwork module to obtain the local dynamic brain functional connectivity hypernetwork training set and test set.

[0119] The training set of the global static brain functional connectivity hypernetwork and the training set of the local dynamic brain functional connectivity hypernetwork are used to train the hypergraph neural network classification model, and the test set of the global static brain functional connectivity hypernetwork and the test set of the local dynamic brain functional connectivity hypernetwork are used to test the classification effect of the hypergraph neural network classification model.

[0120] Step A3, for the same subject, send the training set of the global static brain functional connectivity hypernetwork into the static branch feature learner, and send the training set of the local dynamic brain functional connectivity hypernetwork into the dynamic branch feature learner; use the first feature fusion device to splice the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module for the same subject, and use the feature classifier to predict the brain image category of the subject, and use its true brain image category as the target label to calculate the cross-entropy loss y e is the true brain image label of the e-th sample, p e is the probability that the e-th sample is predicted to be a certain brain image category, e is the training sample number; E is the number of training samples; use the backpropagation algorithm to update the model weights of the feature learner and the feature classifier to improve the ability of the feature learner to learn disease-related brain features.

[0121] Preferably, the hypergraph neural network classification model is also provided with a site classifier; the site classifier is used to predict the detection site to which the subject belongs through the features fused by the first feature fuser; it calculates the cross-entropy loss with the real site as the target label; uses the backpropagation algorithm to update the model weights of the site classifier, so as to improve the ability of the site classifier to distinguish the interference factors caused by site differences from the features extracted by the feature learner.

[0122] Preferably, training the hypergraph neural network classification model may further include the following steps:

[0123] Step A4, splice the features extracted by the dynamic branch feature learner, the static branch feature learner and the individual clinical feature learning module for the same subject, and use the site classifier to predict the detection site to which the subject belongs, and calculate the cross-entropy loss L with its real site as the target label d , F is the number of sites, y e,f is the sign function, which is 1 when the real site label of the e-th sample is f, otherwise 0, p e,f is the probability that the e-th sample predicts the site label as f, and use the backpropagation algorithm to update the model weights of the site classifier, so as to improve the ability of the site classifier to distinguish the interference factors caused by site differences from the features extracted by the feature learner.

[0124] Step A5, splice the features extracted by the dynamic branch feature learner, the static branch feature learner and the individual clinical feature learning module for the same subject, and use the site classifier to predict the detection site to which the subject belongs, and calculate the confusion loss with its real site as the target label Use the backpropagation algorithm to update the model weights of the feature learner to reduce the interference factors caused by site differences in the features extracted by the feature learner.

[0125] Step A6, repeat Step A3 to Step A5 until the model converges, and the overall loss function is L = L p +αL d +βL conf .

[0126] Step A7, the mean and variance of the training set saved in Step A2 can be used for the normalization of the test samples, and the trained hypergraph neural network classification model is tested with the test set; compare the test results with the real values to evaluate the performance of the classification model.

[0127] Preferably, the grid search method can be used to obtain the optimal values of the following parameters: w, s, t, k, α, β and different hidden layer dimension values of GRU.

[0128] The method of grid search can be used to test different values of w, s, t, and k to find the parameters with better classification performance. The larger the window size w, the larger the window step s, the larger the effective threshold t of correlation, and the larger the value of the number k of hyperedge neighbors, the more macroscopic the constructed functional connection hypernetwork is, and vice versa, the more microscopic it is.

[0129] The method of grid search can be used to test different hidden layer dimension values of HGCN and GRU, as well as appropriate values of α and β, to find the parameters with better classification performance. The higher α is, the higher the weight of the site classifier learning site features during model training. The higher β is, the stronger the degree of removing the interference factors of site differences by the feature remover during model training. Their values need to be relatively balanced to achieve a better adversarial training effect.

[0130] The following takes a preferred embodiment of the present invention to further illustrate the working process and working principle of the present invention:

[0131] As Figure 1 shown, the method mainly includes the following steps:

[0132] Step (1), fMRI data acquisition: Acquire the fMRI data of the subjects provided by the REST-meta-MDD dataset, including 1300 patients with major depressive disorder and 1128 patients without major depressive disorder.

[0133] Step (2), Preprocess the acquired Rs-fMRI brain image data through DPARSF software. Specifically, it includes:

[0134] 1) Remove the first 10 time points to make the magnetic field reach a steady state and the subjects adapt to the scanning environment.

[0135] 2) Temporal layer correction.

[0136] 3) Head motion correction: (a) Head motion correction based on the rigid body transformation built in spm. (b) Exclude the subjects with head motion greater than 2 mm and 1 radian; (c) Calculate the frame displacement FD and exclude the subjects with FD>0.2; (d) Subsequent covariate regression reduces the influence of head motion.

[0137] 4) Registration of functional images.

[0138] 5) Remove linear drift and band-pass filtering.

[0139] 6) Regression covariates (white matter signal, cerebrospinal fluid signal, and head motion parameters).

[0140] Step (3), Segment the whole brain tissue into 116 brain regions with different functions according to the physiological AAL template for the preprocessed fMRI brain image data, and extract the average voxel time series of each brain region.

[0141] Step (4.1), the process of constructing hyperedges is as follows Figure 2 As shown, for the time series of each brain region in the same brain image, pairwise calculate the LASSO regression correlation coefficient, and construct the functional connection vector between each brain region and other brain regions based on the correlation to obtain the global static correlation coefficient matrix where N is the number of brain regions and M is the number of hyperedges; based on the sliding window method, according to the fixed time length w and time interval s, divide the fMRI time series of each brain region with a duration of T into segments, calculate the Pearson correlation coefficient of the fMRI time series segments between these brain regions, and obtain a series of local dynamic correlation coefficient matrices P D =(P1,...,P n ),, r is the serial number of the local dynamic correlation coefficient matrix; r = 1, 2,..., n; n is the number of local dynamic correlation coefficient matrices.

[0142] Step (4.2), based on the global static correlation coefficient matrix H of each subject, set the values less than t to 0 to obtain the global static functional connection hypernetwork adjacency matrix of this subject, and then obtain the functional connection hypernetwork G.

[0143] Step (4.3), the r-th local dynamic correlation coefficient matrix of the i-th brain region is P ri , calculate the distance between the i-th brain region and other brain regions according to P ri , and take the first k brain regions with the smallest distance from the i-th brain region as the neighbor nodes of the i-th brain region to construct hyperedges; take all brain regions as the central nodes and repeat this step to obtain a series of local dynamic brain functional connection hypernetworks G D =(G1,...,G q ), v is the serial number of the local dynamic brain functional connection hypernetwork; v = 1, 2,..., q; q is the number of local dynamic brain functional connection hypernetworks.

[0144] Step (4.4): According to the order of each brain region in the brain region division, encode it in the form of a one-hot vector, and encode the position order of the brain region in the sub-functional brain network in the form of a one-hot vector, and splice the two vectors to obtain the hypernetwork node features, that is, the node features corresponding to the i-th brain region are where S is the number of sub-functional brain networks.

[0145] Step (5): The functional connection hypernetwork data is cross-validated by the five-fold cross-validation method to obtain the final hypergraph neural network classification model. The specific description of its algorithm implementation is as follows

[0146] Step (5.1): Assume that the number of samples in the sample dataset is L. Use the five-fold cross-validation method for model training. Given L samples, randomly select L / 5 of them as test samples, and the other 4L / 5 samples as training samples to obtain L / 5 classification test results. Use the average of these L / 5 classification test results to evaluate the model performance.

[0147] The corresponding training sets are called {Train_1, Train_2, …, Train_5}.

[0148] Step (5.2): Perform a normalization operation on the training set to normalize the data into a distribution with a mean of 0 and a variance of 1, and at the same time save the mean and variance of the training set.

[0149] Step (5.3): Use the data saved in step (5.2) for the normalization of test samples. Apply the trained model to the test set to obtain the prediction results of the disease status of the test set. Compare these prediction results with the true values to obtain the discrimination results of this classification. Use the backpropagation algorithm to update the parameters of the feature learner and the feature classifier based on the cross-entropy loss.

[0150] Step (5.4): Apply the trained model to the test set to obtain the prediction results of the site labels of the test set. Compare these prediction results with the true values to obtain the discrimination results of this site prediction. Use the backpropagation algorithm to update the parameters of the site classifier based on the cross-entropy loss.

[0151] Step (5.5): Apply the trained hypergraph neural network classification model to the test set to obtain the prediction results of the site labels of the test set. Compare these prediction results with the true values to obtain the discrimination results of this site prediction. Use the backpropagation algorithm to update the parameters of the site classifier based on the confusion loss.

[0152] Step (5.6): Repeat steps 5.1 - 5.5 until the discrimination results of all samples are calculated, and evaluate the performance of the hypergraph neural network classification model.

[0153] Step (6): Classification parameter tuning based on grid search. Use the grid search method to find the appropriate classification parameters in the hypergraph neural network classification model, including the coefficients w, s, t, k, the different hidden layer dimension values of the first and second hypergraph convolution modules using HGCN, HGCN and GRU, and the appropriate α, β values.

[0154] The specific structure and operation principle of the hypergraph neural network classification model are as Figure 3As shown in the figure. The hypergraph neural network classification model consists of three parts: a feature learner, a feature classifier, and a site classifier. Among them, the feature learner includes a dynamic branch feature learner, a static branch feature learner, and an individual clinical feature learning module;

[0155] The dynamic branch feature learner is used to extract features from a series of local dynamic functional connection hypernetworks of the subject U.

[0156] The static branch feature learner is used to extract features from the global static brain functional connection hypernetwork of the subject U.

[0157] The individual clinical feature learning module is used to extract features from the clinical brain imaging data of the subject U or patients with the same symptoms as the subject U.

[0158] The static branch feature learner includes a first hypergraph convolution module, a first information enhancement module, and a first spatial attention module connected in sequence; the dynamic branch feature learner includes a second hypergraph convolution module, a second spatial attention module, a GRU module, a temporal attention module, a second information enhancement module, and a third spatial attention module.

[0159] Among them, the first and second information enhancement modules include a causal decoupling unit and an orthogonal normalization unit, which are used to reduce data noise and strengthen feature representation to a certain extent.

[0160] The main function of the causal decoupling unit is reflected in improving the interpretability and fairness of the model. Through the causal decoupling unit, the feature representation can be decomposed into multiple independent factors. These changing factors are independent of each other, and each factor represents a certain physical or semantic meaning. In this way, when a single changing factor changes, it will affect the change of a single factor in the generated data, while other factors remain unchanged, thereby improving the interpretability of the representation.

[0161] The orthogonal normalization unit is used to transform a linearly independent vector group into a pairwise orthogonal vector group with a length of 1. Its core includes two steps: orthogonalization and unitarization, aiming to eliminate the redundancy between vectors and improve the calculation efficiency.

[0162] Input the global static functional connection hypernetwork into the static branch, and extract the preliminary feature H through the hypergraph convolutional network s , H s is strengthened through the information enhancement module to obtain H ′ s , and the spatial attention module (trainable weight matrix) is used to further strengthen the feature H ′ s to obtain the static branch feature vector H g , and at the same time obtain the spatial weight vector Z s, which characterizes the influence degree of different brain regions on depression from a global perspective. A series of local dynamic functional connectivity hypernetworks are input into the dynamic branch, and preliminary features H are extracted through the hypergraph convolutional network. d =(H d (1), H d (2),..., H d (n)), and the spatial attention module is used to strengthen H d to obtain Z d . Z d is multiplied by the global spatial weight vector Z s to obtain the feature vector M that integrates the global spatial weight information and local spatial information. At the same time, the H d sequence passes through the GRU module to obtain the temporal feature sequence. It is strengthened by the time attention module (trainable weight matrix) and multiplied by the feature vector M to obtain the feature that integrates the global space, local space, and time information. After is strengthened by the information enhancement module and then strengthened by the spatial attention module to learn the spatial importance degree from the global and local integration perspectives, the final dynamic branch feature H g is obtained. The static branch feature and the dynamic branch feature are concatenated to obtain the final functional connectivity hypernetwork feature. The clinical feature branch is composed of a multi-layer fully connected neural network MLP, and its input is data such as the age, gender, education, head movement, and medication status of the subjects. is concatenated with the individual features learned by the clinical feature branch to obtain the final individual features, which are used as the input data for the subsequent feature classifier and site classifier. The feature classifier and the site classifier are both composed of a multi-layer fully connected neural network MLP. The feature learner is used to learn features from the brain functional connectivity hypernetwork, the feature classifier is used to discriminate the disease status of samples based on the features extracted by the feature learner, and the site classifier is used to discriminate the interference factors caused by site differences in the features extracted by the feature learner.

[0163] In the above text:

[0164] H s is the global static hypergraph deep feature learned by the hypergraph convolution module;

[0165] H ′ s is the global static hypergraph deep feature strengthened by the information enhancement module;

[0166] H g is the hypergraph feature weighted by the spatial attention module;

[0167] Z sis the global static hypergraph spatial attention weight;

[0168] Z d is the local dynamic hypergraph spatial attention weight;

[0169] H d is the local dynamic hypergraph deep feature learned by the hypergraph convolution module;

[0170] is the local dynamic hypergraph temporal feature learned by the GRU module;

[0171] is the local dynamic hypergraph deep feature processed by the time attention module;

[0172] is the final hypergraph feature obtained by concatenating the global static hypergraph final feature and the local dynamic hypergraph final feature.

[0173] After the brain can be divided into 116 brain regions according to the AAL template, the functional magnetic resonance imaging time series containing 175 time points of each subject can be averaged for the voxels within the brain regions. In this way, each brain region can correspond to an average time series, so that each subject obtains 116 average time series of brain regions, corresponding to a matrix of 116×175. Furthermore, the LASSO regression correlation coefficient and Pearson correlation coefficient between brain regions are calculated based on this fMRI data.

[0174] To illustrate the beneficial effects of the method described in the present invention, in the specific implementation process, as Figure 3 shown, we respectively constructed two other models for comparative experiments.

[0175] Let the hypergraph neural network classification model 1 include a feature learner, a first feature fuser, a feature classifier, and a site classifier; the feature learner includes a dynamic branch feature learner, a static branch feature learner, and an individual clinical feature learning module.

[0176] Let the hypergraph neural network classification model 2 include a feature learner, a first feature fuser, a feature classifier; the feature learner includes a dynamic branch feature learner, a static branch feature learner, and an individual clinical feature learning module.

[0177] Let the hypergraph neural network classification model 3 include a feature learner, a first feature fuser, a feature classifier; the feature learner includes a dynamic branch feature learner, a static branch feature learner.

[0178] The dynamic branch feature learner is used to extract features from a series of local dynamic functional connection hypernetworks of subject U.

[0179] The static branch feature learner is used to extract features from the global static brain functional connection hypernetwork of subject U.

[0180] The individual clinical feature learning module is used to extract features from the clinical brain imaging data of subject U or patients with the same symptoms as subject U.

[0181] The first feature fuser is used to fuse the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module.

[0182] The feature classifier is used to classify the features fused by the first feature fuser.

[0183] The site classifier is used to predict the detection site to which subject U belongs through the features fused by the first feature fuser.

[0184] First, we constructed the hypergraph neural network classification model 1 using the method provided by the present invention. Then, we removed the site classifier in the model and the adversarial training step for reducing the interference factors of multi-site differences to form the hypergraph neural network classification model 2. Further, we removed the individual clinical feature learning module in the model to obtain the hypergraph neural network classification model 3. We constructed the same brain functional connection hypernetwork in the hypergraph neural network classification model 1 and the hypergraph neural network classification model 2, and then adopted the same hyperparameters. We carried out model training on the same data set. The experimental results show that the final classification accuracy of the major depressive disorder recognition method using the present invention is 72.84% on 1611 samples (832 patients with depression and 779 healthy subjects). While the classification accuracy of the major depressive disorder recognition method constructed based on model 2 is only 62.69%, and the best classification accuracy of model 3 under the same conditions is only 62.13%. The detailed classification performance of each model is shown in the following table. It can be seen that the brain imaging hypergraph processing system integrating brain functional imaging and clinical data proposed by the present invention has obtained the best results in various indicators. And after removing the multi-site difference reduction mechanism and the individual clinical feature integration mechanism, the classification performance of the brain imaging hypergraph processing system integrating brain functional imaging and clinical data has decreased, further proving the effectiveness of multi-site difference reduction and individual clinical feature integration for disease recognition.

[0185] Model Accuracy (%) Precision (%) Recall (%) AUC Model 1 72.84 70.30 75.06 0.77 Model 2 62.69 64.24 63.70 0.69 Model 3 62.13 64.26 60.58 0.67

[0186] Function modules such as the above-mentioned feature learner, first feature fusion unit, feature classifier, dynamic branch feature learner, static branch feature learner, individual clinical feature learning module, first hypergraph convolution module, first information enhancement module, first spatial attention module, second hypergraph convolution module, second spatial attention module, GRU module, temporal attention module, second information enhancement module, third spatial attention module, GRU module, first feature fusion unit, second feature fusion unit, feature classifier, site classifier, multi-layer fully-connected neural network MLP, etc. can all adopt applicable function modules in the prior art, or adopt applicable function modules and software systems in the prior art, and be constructed by conventional technical means.

[0187] The above-described embodiments are only used to illustrate the technical ideas and features of the present invention, and the purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The patent scope of the present invention cannot be limited only by these embodiments. That is, any equivalent changes or modifications made to the spirit disclosed by the present invention still fall within the patent scope of the present invention.

Claims

1. A brain imaging hypergraph processing system that integrates brain functional imaging and clinical data, characterized in that, It includes a global static brain functional connectivity hypernetwork module, a local dynamic brain functional connectivity hypernetwork module, and a hypergraph neural network classification model. Randomly select a subject from the set of subjects, and this subject is called subject U. The global static brain functional connectivity hypernetwork module constructs a functional connection vector between each brain region and other brain regions based on the correlation of the brain imaging data of subject U in the order of brain regions, generating a global static correlation coefficient matrix. Then, based on the global static correlation coefficient matrix, it generates a global static functional connectivity hypernetwork adjacency matrix. Further, it generates the global static brain functional connectivity hypernetwork of this subject. The local dynamic brain functional connectivity hypernetwork module divides the brain imaging time series data of each brain region of subject U into multiple segments based on the sliding window method, calculates the Pearson correlation coefficients of these segments, generates a series of local dynamic correlation coefficient matrices of brain regions, and then generates a series of local dynamic brain functional connectivity hypernetworks of this subject from each local dynamic correlation coefficient matrix of each brain region. The hypergraph neural network classification model includes a feature learner, a first feature fusion module, and a feature classifier. The feature learner includes a dynamic branch feature learner, a static branch feature learner, and an individual clinical feature learning module. The dynamic branch feature learner is used to extract features from a series of local dynamic functional connectivity hypernetworks of subject U. The static branch feature learner is used to extract features from the global static brain functional connectivity hypernetwork of subject U. The individual clinical feature learning module is used to extract features from the clinical brain imaging data of subject U or patients with the same symptoms as subject U. The first feature fusion module is used to fuse the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module. The feature classifier is used to classify the features fused by the first feature fusion module.

2. The brain image hypergraph processing system that integrates brain functional imaging and clinical data according to claim 1, wherein The hypergraph neural network classification model also includes a site classifier. The site classifier is used to predict the detection site to which subject U belongs through the features fused by the first feature fusion module. It calculates the cross-entropy loss with the real site as the target label. It uses the backpropagation algorithm to update the model weights of the site classifier to enhance the ability of the site classifier to distinguish interference factors caused by site differences from the features extracted by the feature learner.

3. The brain imaging hypergraph processing system integrating brain functional imaging and clinical data according to claim 1, wherein The static branch feature learner includes a first hypergraph convolution module, a first information enhancement module, and a first spatial attention module connected in sequence. The dynamic branch feature learner includes a second hypergraph convolution module, a second spatial attention module, a GRU module, a temporal attention module, a second information enhancement module, and a third spatial attention module. The first and second hypergraph convolution modules are used to update the features of nodes by simulating the propagation and aggregation of information in hyperedges. The first and second information enhancement modules are used to reduce data noise and strengthen feature representation. The first and second spatial attention modules are used to enhance the sensitivity to specific spatial regions in the input data. The GRU module is used to extract sequence data features. The temporal attention module is used to enhance the sensitivity to specific sequence regions in the input time series data. The output end of the second hypergraph convolutional module is respectively connected to the input ends of the second spatial attention module and the GRU module; The output ends of the second spatial attention module and the GRU module are connected to the input end of the temporal attention module; The output end of the temporal attention module is connected to the input end of the second information enhancement module; The output end of the second information enhancement module is connected to the input end of the third spatial attention module; the output ends of the second and third spatial attention modules are connected to the input end of the first feature fusion unit.

4. The brain imaging hypergraph processing system integrating brain functional imaging and clinical data according to claim 3, characterized in that, The dynamic branch feature learner further includes a second feature fusion unit; The second feature fusion unit is used to fuse the features output by the first and second spatial attention modules; The input end of the second feature fusion unit is connected to the output ends of the first and second spatial attention modules; The output end of the second feature fusion unit is connected to the input end of the temporal attention module.

5. A method for processing brain image hypergraph by fusing brain functional imaging and clinical data of a brain image hypergraph processing system using the one described in claim 1, characterized in that, Collect and preprocess the resting-state fMRI data of a large number of cross-site subjects, use the brain regions marked by the AAL template as the regions of interest, and obtain the average time series of each brain region; by the global static brain functional connectivity hypernetwork module, based on the brain imaging data of each region of each subject, use the regression method to construct a global static brain functional connectivity hypernetwork; by the local dynamic brain functional connectivity hypernetwork module, use the sliding window method and the Pearson correlation coefficient method to construct a series of local dynamic brain functional connectivity hypernetworks of the subject based on the time series of each brain region of each subject; by the individual clinical feature learning module, extract features from the clinical brain imaging data of the subject or patients with the same symptoms as the subject; by the dynamic branch feature learner, extract features from a series of local dynamic functional connectivity hypernetworks of the subject; by the static branch feature learner, extract features from the global static brain functional connectivity hypernetwork of the subject; by the first feature fusion unit, fuse the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module, and classify the features fused by the first feature fusion unit by the feature classifier.

6. The brain image hypergraph processing method for integrating brain functional imaging and clinical data according to claim 5, wherein The method includes the following method steps: Calculate the LASSO regression correlation coefficients of the fMRI time series between brain regions for each subject, obtain the correlations between brain regions, and construct a functional connection vector between each brain region and other brain regions based on the correlations in the order of brain regions to obtain the global static correlation coefficient matrix where N is the number of brain regions and M is the number of hyperedges; Based on the global static correlation coefficient matrix H of each subject, set the matrix element values less than t to 0. After H is filtered, the global static functional connectivity hypernetwork adjacency matrix of the subject is obtained, where each column vector is a hyperedge. Let i be the brain region number, and let the central node of the i-th column vector be the i-th brain region, and other nodes be the neighbor nodes in this hyperedge. The correlation coefficient between it and the central node is the correlation between brain regions. All hyperedges form a global static brain functional connectivity hypernetwork G; Based on the sliding window method, according to a fixed time length \(w\) and time interval \(s\), the fMRI time series of each brain region with a duration of \(T\) is divided into segments, and the Pearson correlation coefficients of the fMRI time series segments between these brain regions are calculated to obtain a series of local dynamic correlation coefficient matrices \(P\ D , \(P\ D =(P_1,\cdots,P\ n ), where \(r\) is the serial number of the local dynamic correlation coefficient matrix; \(r = 1, 2, \cdots, n\); \(n\) is the number of local dynamic correlation coefficient matrices; The r-th local dynamic correlation coefficient matrix of the i-th brain region is P ri , according to P ri Calculate the distance between the i-th brain region and other brain regions, and take the top k brain regions with the smallest distance from the i-th brain region as the neighbor nodes of the i-th brain region to construct a hyperedge; this hyperedge takes the i-th brain region as the central node, and other nodes as the neighbor nodes in the hyperedge; repeat this step for all brain regions as the central nodes to obtain a series of local dynamic brain functional connection hypernetworks G D =(G1,...,G q ), v is the serial number of the local dynamic brain functional connection hypernetwork; v = 1, 2,..., q; q is the number of local dynamic brain functional connection hypernetworks; Encode according to the order of each brain region in the brain region division in the form of a one-hot vector, and encode the position order of the brain region in the sub-functional brain network in the form of a one-hot vector. Concatenate the two vectors to obtain the hypernetwork node feature, that is, the node feature corresponding to the i-th brain region is where S is the number of sub-functional brain networks, and thus the position information of different dimensions of the brain region is correspondingly incorporated into the hypergraph features of the local dynamic brain functional connection hypernetwork and the global static brain functional connection hypernetwork.

7. The method for processing brain imaging hypergraph by integrating brain functional imaging and clinical data according to claim 6, characterized in that, Use the following method steps to train the hypergraph neural network classification model: Step A1, collect and preprocess the resting-state fMRI data of the subject, and add individual data such as education, age, gender, and medical conditions corresponding to the subject; Randomly divide the preprocessed fMRI data into a training set and a test set, normalize the feature data in the training set to a distribution with a mean of 0 and a variance of 1, and save the mean and variance of the training set; Step A2, take different values of w, s, t, and k, and process the training set and the test set by the global static brain functional connectivity hypernetwork module to correspondingly obtain the global static brain functional connectivity hypernetwork training set and the test set; process the training set and the test set by the local dynamic brain functional connectivity hypernetwork module to correspondingly obtain the local dynamic brain functional connectivity hypernetwork training set and the test set. The training set of the global static brain functional connectivity hypernetwork and the training set of the local dynamic brain functional connectivity hypernetwork are used to train the hypergraph neural network classification model, and the test set of the global static brain functional connectivity hypernetwork and the test set of the local dynamic brain functional connectivity hypernetwork are used to test the classification effect of the hypergraph neural network classification model. Step A3: For the same subject, send the training set of the global static brain functional connectivity hypernetwork into the static branch feature learner, and send the training set of the local dynamic brain functional connectivity hypernetwork into the dynamic branch feature learner; use the first feature fusion device to splice the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module for the same subject, and use the feature classifier to predict the brain image category of this subject, and calculate the cross-entropy loss using its true brain image category as the target label. y e is the true brain image label of the e-th sample, and p e is the probability that the e-th sample is predicted as a certain brain image category, where e is the training sample number; E is the number of training samples; use the backpropagation algorithm to update the model weights of the feature learner and the feature classifier to improve the ability of the feature learner to learn disease-related brain features.

8. The brain image hypergraph processing method for integrating brain functional imaging and clinical data according to claim 7, wherein The hypergraph neural network classification model also has a site classifier; the site classifier is used to predict the detection site to which the subject belongs through the features fused by the first feature fuser; it calculates the cross-entropy loss with the true site as the target label; uses the backpropagation algorithm to update the model weights of the site classifier to improve the ability of the site classifier to distinguish the interference factors caused by site differences from the features extracted by the feature learner.

9. The method for processing brain image hypergraph integrating brain functional imaging and clinical data according to claim 8, wherein Training the hypergraph neural network classification model also includes the following steps: Step A4, splice the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module for the same subject, and use the site classifier to predict the detection site to which the subject belongs. Calculate the cross-entropy loss L with its true site as the target label d , F is the number of sites, y e,f is the sign function, which is 1 when the true site label of the e-th sample is f, otherwise 0, p e,f is the probability that the predicted site label of the e-th sample is f. Use the backpropagation algorithm to update the model weights of the site classifier to improve the ability of the site classifier to distinguish the interference factors caused by site differences from the features extracted by the feature learner; Step A5, splice the features extracted by the dynamic branch feature learner, the static branch feature learner, and the individual clinical feature learning module for the same subject, and use the site classifier to predict the detection site to which the subject belongs, and calculate the confusion loss using its true site as the target label. Use the backpropagation algorithm to update the model weights of the feature learner to reduce the interference factors caused by site differences in the features extracted by the feature learner. Step A6, repeat Step A3 to Step A5 until the model converges. The overall loss function is L = L p + αL d + βL conf .

10. The brain image hypergraph processing method for fusing brain functional imaging and clinical data according to claim 9, characterized in that, By means of grid search, obtain the optimal values of the following parameters: w, s, t, k, α, β, and different hidden layer dimension values of GRU.

Images