A brain network classification method based on community detection and dual-path autoencoding

Abstract

The present invention discloses a brain network classification method based on community detection and dual-path autoencoding. We introduce a community detection method to detect the potential community structure in the brain network, and then, according to the result of the community detection, readjust the sorting order of the nodes in the network. On the one hand, it allows the modular structure in the network to be retained, and on the other hand, the adjusted brain network is more suitable for the rectangular convolution kernel in the deep learning method. Since brain network data has two modes, namely structural mode and functional mode. In order to better integrate the information of the two modes, the present invention adopts dual-path autoencoding technology to classify brain networks. First, we use the structural mode to guide the community detection and network structure adjustment method of the functional mode network. Then, the dual-path autoencoding technology with an attention mechanism is used to extract important high-order information in the network. Finally, the complementary information of the structural mode and the functional mode is comprehensively utilized to complete the classification task of the brain network.

Description

Technical Field

[0001] The present invention belongs to the field of machine learning, and in particular relates to a brain network classification method based on community detection and dual-path autoencoding. Background Art

[0002] In recent years, neuroimaging technology has developed rapidly, bringing us various forms of brain imaging data, providing great support for our study of the internal organization and activities of the human brain. Therefore, brain network analysis methods that rely on neuroimaging technology have received more and more attention and have gradually become mainstream. In addition, related research work has shown that brain network analysis methods can achieve good results. In brain network analysis research, the most commonly used imaging technologies are functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). fMRI can reflect the temporal correlation between BLOD signals in brain regions, and the functional connection network of the brain can be constructed through fMRI data. DTI can reveal the physical connection between functionally related gray matter areas, thereby constructing the structural connection network of the brain. Whether it is single-modal network analysis or multi-modal network analysis, finding a suitable method to explore the potential internal structure of the brain network and extract effective features to improve the accuracy of brain network classification are key tasks.

[0003] However, there are several difficulties in brain network analysis. 1) Noise: In the process of medical image acquisition, it is easy to be disturbed by the objective environment, so there is often a lot of noise. For example, the data collected from the same subject at different times of the day may be different, which brings difficulties to our analysis task. 2) Highly nonlinear: Graph structure data often have potential highly nonlinear relationships, and brain networks, as a typical graph structure data, also have certain nonlinear relationships inside. How to capture nonlinear relationships in brain networks has always been a research hotspot; 3) Structure preservation: Many related studies have shown that there are often some structures inside brain networks, such as small worlds, modules, and community structures, which play an important role in the process of brain network analysis. However, many traditional brain network analysis methods will pull the matrix representing the network into a vector, losing this important structural information. It is important to maintain the community structure while learning brain network embedding; 4) Multimodal network analysis: At present, multimodal brain network analysis combining functional and structural modalities has become very popular. Since the characteristics of structural networks and functional networks are different, we cannot simply use the same method to analyze them or directly fuse them when analyzing multimodal brain networks. How to consider both structural connectivity and functional connectivity in multimodal brain network analysis tasks has always been a key issue.

[0004] Many methods have been proposed to solve these difficulties. Typical ones include subgraph mining methods and graph kernel methods. Graph kernel methods usually calculate the similarity between two networks based on substructures, such as shortest paths, random walks, or other subgraphs. Graph kernel methods can be used to integrate local connectivity or global topological properties of brain networks. However, since the models built using graph kernel methods are shallow models, graph kernel methods can only extract shallow features, and their feature learning ability is weak in complex situations, so they cannot effectively capture the complex and highly nonlinear relationships that are potentially present in brain networks. Subgraph mining methods are unsupervised methods that aim to extract useful subgraph features from a set of graphs by adopting some specific criteria. The current popular approach is to select frequent substructures by connecting subgraph patterns. However, due to the huge complexity of brain network data, this method is difficult to accurately capture the highly nonlinear characteristics of brain networks. In addition, it is impossible to detect the structural information inside the network.

[0005] In recent years, deep learning methods have become very popular and have achieved good results in natural image processing and natural language processing. Therefore, many scholars have introduced deep learning methods into medical image analysis. However, deep learning methods are not suitable for directly acting on brain networks. Because in deep learning methods, rectangular convolution kernels are generally used. This rectangular convolution kernel is suitable for capturing relationships in Euclidean space. Natural images satisfy such a Euclidean domain, so deep learning has achieved great results in natural image research. However, unlike natural images, in brain networks, the connections between different brain regions are located on irregular or non-Euclidean domains. This type of edge is a common representation of heterogeneous pairwise relationships. Therefore, the receptive field of the rectangular convolution kernel cannot encode the module structure in the connection matrix well, and it is difficult to identify the same-sex patterns and difference patterns between different samples. In order to use deep learning methods to study brain network data, it is necessary to adjust the rectangular convolution kernel or adjust the brain network structure. In addition, early brain network research often used single-modal data to construct brain networks. In recent years, multimodal network analysis methods that integrate structural networks and functional networks have become increasingly popular and have achieved good results. However, although traditional methods can fully integrate information from multiple modalities, they can only consider the shallow characteristics of brain networks and cannot perform multimodal integration at a high level. In order to address the limitations of traditional methods, more and more work has introduced deep learning methods into multimodal brain network analysis tasks. Summary of the invention

[0006] Purpose of the invention: In order to overcome the problem that it is difficult to retain modular information in brain networks and that deep learning methods cannot directly act on brain network data, the present invention provides a brain network classification method based on community detection and dual-path autoencoding.

[0007] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:

[0008] A brain network classification method based on community detection and two-way autoencoding includes the following steps:

[0009] (1) Collect structural modality data DTI and functional modality data fMRI of the human brain.

[0010] (2) Constructing structural modality brain network W based on DTI data s , constructing a functional modality brain network W based on fMRI data f .

[0011] (3) The method of using structural modal data to guide functional modal data is used to detect the potential community structure in the brain network.

[0012] (4) Based on the detected community structure, the arrangement order of nodes in the structural brain network and the functional brain network is readjusted so that they can retain the modular information in the network and adapt to the subsequent matrix convolution kernel.

[0013] (5) Design a dual-path autoencoder model with an attention mechanism, and send the structural network and functional network after node reordering into their respective autoencoder channels to extract high-order nonlinear important information, which is represented by the hidden layer.

[0014] (6) Comprehensively utilize the latent information of structural modes and functional modes to complete the brain network classification task.

[0015] Furthermore, the specific method of constructing the structural modal network and the functional modal network in step (2) is:

[0016] For functional modality data, the Pearson similarity of blood signals between two brain regions is calculated, and then regularized to obtain the functional brain network W f For structural modal data DTI, it is necessary to propose the number of physical fibers between each two brain regions and construct the structural brain network W s .

[0017] Furthermore, the specific method of using the structural modality network to guide the functional modality network in step (3) to detect the community structure in the brain network is:

[0018] Using the spectral clustering method, we find a suitable community division rule on the structural brain network, and use this rule to divide the structural network into several subgraphs, so that the sum of the node weights within the subgraph is high, and the sum of the node weights between subgraphs is low. Then, this subgraph division rule is applied to the functional brain network to find the community structure of the functional brain network. Since there are many parameters in the community detection process, and different parameters will result in different subgraph division structures, we will jointly train the parameters of step 3 and the subsequent dual-path autoencoder classification model to obtain the best community division rule that can improve classification accuracy. At the same time, this rule comprehensively considers the structural modal and functional modal network information, rather than relying solely on the structural modal network. Suppose the brain network is divided into k non-overlapping communities using the spectral clustering method: A 1 , A 2 , ..., A k The weight between any two subgraphs A and B is defined as: W(A, B) = ∑ i∈A，j∈B w ij . Then for the set of k subgraph points, there is in Yes A i The complement of . Since the principle of using spectral clustering to divide communities is to make the sum of the node weights within the subgraph high and the sum of the node weights between subgraphs low. Therefore, the optimization goal is:

[0019]

[0020] Among them, vol(A i ) means A i The sum of the weights of the points within the subgraph. By solving this objective function, we can get a set of rules for subgraph partitioning, that is, the rules for community partitioning.

[0021] Furthermore, the specific method of adjusting the order of nodes in the brain network according to the community structure in step (4) is:

[0022] The network nodes belonging to the same community are placed in adjacent positions in the brain network, while the nodes belonging to different communities are placed separately. The same method is used for both structural brain networks and functional brain networks. After adjustment, the brain network has the property of approximating block diagonals. A diagonal block corresponds to a community. The diagonal blocks are distributed with nodes that are highly densely connected in the community, while the non-diagonal blocks are distributed with nodes that have low connectivity with other nodes in the network. The structural brain network after adjustment is represented by W′ s , the adjusted functional brain network is represented by W′ f To facilitate the subsequent description, W (s) Replace W′ s , use W (f) Replace W′ f.

[0023] Furthermore, the specific method of designing a dual-path autoencoder model with an attention mechanism in step (5) is:

[0024] Each autoencoder model has two basic structures, namely the encoder and the decoder. The structure of the encoder is designed as follows: the first convolutional layer, the first maximum pooling layer, the second convolutional layer, the second maximum pooling layer, and the fully connected layer. The structure of the decoder is designed as follows: the fully connected layer, the first deconvolutional layer, the second deconvolutional layer, and the first convolutional layer. Taking the functional modality network as an example, we will use the functional modality network W after adjusting the node order. (f) As the input of the encoder, the intermediate representation layer feature h is obtained after training (f) Then, the output hidden layer of the encoder is used as the input of the decoder and sent to the decoder for training to obtain the reconstructed functional brain network. The encoding-decoding process of the structural modality brain network is the same as that of the functional modality, and it is trained simultaneously with the encoding-decoding model of the functional modality. (s) , after the encoding stage, the hidden layer h is obtained (s) , and then the reconstructed structure network is obtained through the decoding stage Our goal is to make the reconstructed functional network as similar as possible to the original input functional network, and the reconstructed structural network as similar as possible to the original input structural network, so as to obtain the most representative hidden layer features. Therefore, our loss function in the encoding-decoding stage is:

[0025]

[0026] Among them, N is the number of samples.

[0027] Generally speaking, the areas in the receptive field of the convolution kernel are divided into different levels of importance. In the receptive field, some areas often contribute more to the classification task, that is, they are more important, while the remaining areas contribute less to the classification task, that is, they are less important. Therefore, in the brain network classification task, a more reasonable approach is to focus on the areas with high importance. There are many convolution kernels inside the autoencoder model. In order to hope that the model can distinguish the importance of the areas in the receptive field in the process of learning the brain network, improve the effect of feature selection and the accuracy of the final classifier, we introduce the self-attention mechanism into the model. The specific method of using the self-attention mechanism is:

[0028] Assume that the current feature map is represented by matrix X. After the self-attention calculation, an attention map matrix can be obtained. The meaning of each element of the attention map matrix can be understood as "weight". Its i-th row element represents the influence of all pixels at all positions in the feature map on the i-th pixel. Then, multiply this attention map matrix by the input feature map to obtain the feature map weighted by attention, represented by Y. It should be noted that here we do not directly output this Y, but introduce the idea of ​​residual network: Y = γ * Y + X, where γ is a parameter, indicating that in this formula, the weight of the feature map after the overall attention is applied needs to be updated through back propagation. X is the original input feature map. In the initial stage, the value of γ is 0, and the input feature map is directly returned. Afterwards, with learning, the feature map weighted by attention is gradually added to the original feature map, thereby emphasizing the part where attention is applied. In the design process of our autoencoder model, we introduced a self-attention layer in the hope that the model can better consider the importance of the area within the receptive field during the learning process, so as to extract more suitable features and improve the accuracy of the classification model. The relevant experimental results show that in our autoencoder model, placing this self-attention layer at the end of the encoding stage works best.

[0029] Furthermore, the specific method of step (6) to comprehensively utilize the hidden layer information of the structural mode and the functional mode to complete the brain network classification task is:

[0030] The hidden layer of the autoencoder model has a good representation learning ability, which represents the important high-order information in the original input data. In order to comprehensively utilize the information of the two modes during classification, we concatenate the hidden layers of the functional mode and the structural mode to obtain h, and map it to a low-dimensional space through a fully connected layer, and then predict the label. It is worth noting that since the hidden layer of the structural mode and the hidden layer of the functional mode may be distributed in two different spaces, and we need to combine the information of the two hidden layers to predict the label, it is not reasonable to simply concatenate the two vectors distributed in different spaces. In order to solve this problem, we introduce MMD loss into the model to make the two spaces as similar as possible. MMD (Maximum Mean Discrepancy) can measure the similarity of two spaces. The larger the value calculated by MMD loss, the greater the distribution difference between the two spaces. If the calculated value is small enough, it can be considered that the two spaces have the same distribution. In our experiment, of course, we hope that the distribution of the structural mode hidden layer and the distribution of the functional mode hidden layer are as close as possible, so we hope to minimize the MMD loss of the two hidden layer spaces, which can be expressed as follows:

[0031]

[0032] Here, we choose Gaussian kernel as the kernel function, so our MMD loss is expressed as

[0033]

[0034] In the classification stage, cross loss entropy is selected as the loss function of the classifier. Therefore, the loss function of the classification stage is defined as

[0035]

[0036] Finally, the loss function of the entire model is obtained:

[0037] Loss sum =Loss rec +αLoss cls +βLoss mmd

[0038] Among them, α and β are weight parameters. When training the model, we aim to minimize the total loss function, learn the best model parameters and hidden layer representation, and then combine the features of the two hidden layers to complete the classification task of the brain network. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is the overall framework diagram of the brain network classification method based on community detection and dual-path autoencoding;

[0040] Figure 2 It is a workflow diagram for detecting functional modal network communities and adjusting brain network structures based on structural modalities.

[0041] Figure 3 This is the workflow diagram of the self-attention mechanism;

[0042] Figure 4 It is the workflow diagram of the dual-path autoencoder model. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings.

[0044] like Figure 2The figure shows the workflow of community detection of functional modality network and brain network structure adjustment based on structural modality guidance. First, the spectral clustering method is applied to the structural modality network to detect the potential community structure of the structural modality network. Then, a rule for reordering network nodes is obtained based on the detected community structure, and the order of nodes in the network is adjusted using this rule. The principle of adjustment is: if two nodes belong to the same community, they are placed in adjacent positions of the network, and nodes belonging to different communities are placed separately. After adjustment, the network will have the property of approximate block diagonal. Each diagonal block corresponds to a community, and the nodes in the community are distributed in the diagonal block. After adjustment, not only can the potential modular structure information in the network be retained, but also the brain network can be made suitable for rectangular convolution kernels. Since there are two modal data, a simple method is to process the two modal data separately: apply spectral clustering to the structural modal network to obtain the community division rule of the structural modal network, and use this rule to guide the adjustment of the structural modal network; apply spectral clustering to the functional modal network to obtain the community division rule of the functional modal network, and use this rule to guide the adjustment of the functional modal network. However, this approach is not very reasonable because it finds two community division rules for the same person's brain. Secondly, it cannot make good use of the complementary information between the two modalities. Therefore, the method of structural mode guiding functional mode is adopted: spectral clustering is applied to the structural mode network to obtain the community division rule of the network, and then the structural mode network and the functional mode network are adjusted to reorder the network nodes according to this rule. On the one hand, structural mode is the basis of functional mode, so it is reasonable to do so. On the other hand, using this method, only one community structure is found for each sample brain, which is also reasonable. In addition, since the parameters have an impact on the detection results during the community detection process, different parameters will result in different community detection results. Therefore, we jointly train the learning parameters of this process and the subsequent classification model. In this way, the community structure finally found can comprehensively consider the information of the two modalities and can truly improve the classification accuracy, rather than relying solely on the structural mode network.

[0045] The specific implementation details of using spectral clustering to detect community structure in the network are introduced as follows: Assume that the brain network is divided into k disjoint communities using spectral clustering: A 1 , A 2 , ..., A k The weight between any two subgraphs A and B is defined as: W(A, B) = ∑ i∈A，j∈B w ij . Then for the set of k subgraph points, there is in Yes A iThe complement of . Since the principle of spectral clustering to divide communities is to make the sum of the weights of nodes within a subgraph high and the sum of the weights of nodes between subgraphs low. Therefore, the optimization goal is:

[0046]

[0047] Among them, vol(A i ) means A i The weight sum of the points in the subgraph. In order to simplify the solution, the indicator vector h is introduced

[0048]

[0049] Among them, v i represents the i-th brain region node in the brain network. in is the network connection of all samples w mn The average value of . So the objective function is transformed into

[0050]

[0051] in is the average value of the brain network connection matrix W of all samples, and D is the degree matrix. After solving the objective function, the community division rule of the brain network can be obtained. After obtaining this rule, it is used to analyze the structural network W. s and the functional network W f Node reordering. The adjusted structured brain network is represented as W′ s , the adjusted functional brain network is represented by W′ f To facilitate the subsequent description, W (s) Replace W′ s , use W (f) Replace W′ f .

[0052] like Figure 3The following is the workflow of the self-attention mechanism. First, assume that the current feature map is represented by matrix X, and convolve it with three 1x1 convolution kernels to obtain proj_query, proj_key, and proj_value. Then apply matrix multiplication to proj_query and proj_key to obtain the energy matrix, which reflects the influence relationship between two pixels. Then use softmax to normalize it row by row to obtain the attention map matrix. The meaning of each element of the attention map matrix can be understood as "weight". Its i-th row element represents the influence of all pixels at all positions in the feature map on the i-th pixel. Then, multiply this attention map matrix with proj_value to obtain the feature map weighted by attention, represented by Y. It should be noted that γ is not directly output here, but the idea of ​​residual network is introduced: Y = γ*Y+X, where γ is a parameter, indicating that in this formula, the weight of the feature map after the overall attention is applied needs to be updated through back propagation. X is the original input feature map. In the initial stage, the value of γ is 0, and the input feature map is directly returned. Later, as learning progresses, the feature map weighted by attention is gradually added to the original feature map, thereby emphasizing the part to which attention is applied. Relevant experimental results show that in the autoencoder model, placing this self-attention layer at the end of the encoding stage works best.

[0053] like Figure 4 This is the workflow diagram of the dual-path autoencoder model. Each autoencoder model has two basic structures, namely the encoder and the decoder. The structure of the encoder is designed as follows: the first convolution layer, the first activation function, the first maximum pooling layer, the second convolution layer, the second activation function, the second maximum pooling layer, the attention layer, and the fully connected layer. The structure of the decoder is designed as follows: the fully connected layer, the first deconvolution layer, the first activation function, the second deconvolution layer, the second activation function, the first convolution layer, and the third activation function. The first and second activation functions of the encoder module are both Relu; the convolution kernel size of the first and second convolution layers is 5*5; the size of the first and second maximum pooling layers is 2*2. The first, second, and third activation functions of the decoder module are all Relu, the convolution kernel size of the first and second deconvolution layers is 6*6; the convolution kernel size of the first convolution layer is 3*3. Taking the functional modality network as an example, the functional modality network W after node order adjustment is (f) As the input of the encoder, the intermediate representation layer feature h is obtained after training (f)(Hidden layer features). Then, the output hidden layer of the encoder is used as the input of the decoder and sent to the decoder for training to obtain the reconstructed functional brain network. The encoding-decoding process of the structural modality brain network is the same as that of the functional modality, and it is trained simultaneously with the encoding-decoding model of the functional modality. (s) , after the encoding stage, the hidden layer h is obtained (s) , and then the reconstructed structure network is obtained through the decoding stage The goal of the model is to make the reconstructed functional network as similar as possible to the original input functional network, and the reconstructed structural network as similar as possible to the original input structural network, so as to obtain the most representative hidden layer features. Therefore, the loss function of the model encoding-decoding stage is:

[0054]

[0055] Among them, N is the number of samples.

[0056] Since the hidden layer of the structural mode and the hidden layer of the functional mode may be distributed in two different spaces, and the information of the two hidden layers is needed to predict the label, it is not reasonable to simply concatenate the two vectors distributed in different spaces. In order to solve this problem, MMD loss is introduced into the model to make the two spaces as similar as possible. MMD (Maximum Mean Discrepancy) can measure the similarity of two spaces. The larger the value calculated by MMDloss, the greater the distribution difference between the two spaces. If the calculated value is small enough, it can be considered that the two spaces have the same distribution. In the experiment, the distribution of the structural mode hidden layer and the distribution of the functional mode hidden layer are as close as possible, so we hope to minimize the MMD loss of the two hidden layer spaces, which can be expressed as follows:

[0057]

[0058] Here, the Gaussian kernel is selected as the kernel function. Therefore, the MMD loss is expressed as

[0059]

[0060] In the classification stage, cross loss entropy is selected as the loss function of the classifier. Therefore, the loss function of the classification stage is defined as

[0061]

[0062] Finally, the loss function of the entire model is obtained:

[0063] Loss sum =Lossrec +αLoss cls +βLoss mmd

[0064] Among them, α and β are weight parameters. When training the model, the goal is to minimize the total loss function, learn the best model parameters and hidden layer representation, and then combine the features of the two hidden layers to complete the classification task of the brain network.

[0065] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A brain network classification method based on community detection and two-way autoencoding, characterized by: The following steps are involved: (1) Collect structural modality data DTI and functional modality data fMRI of the human brain; (2) Constructing structural modality brain network W based on DTI data s , constructing a functional modality brain network W based on fMRI data f ; For functional modality data, the Pearson similarity of blood signals between two brain regions is calculated, and then regularized to obtain the functional modality brain network W. f For structural modal data DTI, it is necessary to propose the number of physical fibers between each two brain regions and construct the structural modal brain network W s ; (3) Using structural modal data to guide functional modal data, the potential community structure in the brain network is detected; The specific method is as follows: using the spectral clustering method, first find a suitable community division rule on the structural modal brain network, and use this rule to divide the structural modal brain network into several subgraphs, so that the sum of the node weights within the subgraph is high, and the sum of the node weights between subgraphs is low; then, apply this subgraph division rule to the functional modal brain network to find the community structure of the functional modal brain network; Since there are many parameters in the community detection process, and different parameters will result in different subgraph division structures, this step is combined with the subsequent dual-path autoencoder classification model to train parameters to obtain the best community division rule that can improve classification accuracy; at the same time, this rule comprehensively considers the information of the structural modal brain network and the functional modal brain network, rather than relying solely on the structural modal brain network; assuming that the structural modal brain network is divided into k non-overlapping communities using the spectral clustering method: A1, A2, …, A k , define the weight between any two subgraphs A and B as: W(A,B)=∑ i∈A,j∈B w ij , then for the set of k subgraph points, there is in Yes A i The complement of Since the principle of using spectral clustering to divide communities is to make the sum of node weights within a subgraph high and the sum of node weights between subgraphs low, the optimization goal is: By solving this objective function, we can obtain a set of subgraph partitioning rules, that is, community partitioning rules; (4) According to the detected community structure, the arrangement order of nodes in the structural modality brain network and the functional modality brain network is readjusted so that the modular information in the network can be retained and adapted to the subsequent matrix convolution kernel; The specific method of adjusting the order of nodes in the brain network according to the community structure is: placing network nodes belonging to the same community in adjacent positions in the brain network, and placing nodes belonging to different communities separately. The same method is used for both structural modal brain networks and functional modal brain networks. After adjustment, the brain network has the property of approximate block diagonal, where a diagonal block corresponds to a community. The diagonal blocks are distributed with nodes that are highly densely connected in the community, while the non-diagonal blocks are distributed with nodes in the brain network that have low connectivity with other nodes. The structural modal brain network after adjustment is represented by W' s , the adjusted functional modality brain network is represented as W' f , for the convenience of subsequent description, W (s) Replace W' s , use W (f) Replace W' f ; (5) Design a dual-path autoencoder model with an attention mechanism, and send the structural modality brain network and the functional modality brain network after node reordering into their respective autoencoder channels to extract high-order nonlinear important information and represent it with hidden layers; The specific method for designing a two-way autoencoder model with an attention mechanism is: Each autoencoder model has two basic structures, namely the encoder and the decoder. The structure of the encoder is designed as follows: the first convolutional layer, the first maximum pooling layer, the second convolutional layer, the second maximum pooling layer, and the fully connected layer; The structure of the decoder is designed as follows: fully connected layer, first deconvolution layer, second deconvolution layer, first convolution layer; Taking the functional modality brain network as an example, the functional modality brain network W after node order adjustment is (f) As the input of the encoder, the intermediate representation layer feature h is obtained after training (f) Then, the output hidden layer of the encoder is used as the input of the decoder and sent to the decoder for training to obtain the reconstructed functional modality brain network. The encoding-decoding process of the structural modality brain network is the same as that of the functional modality brain network, and it is trained simultaneously with the encoding-decoding model of the functional modality brain network; the original structural modality brain network W (s) , after the encoding stage, the hidden layer h is obtained (s) , and then the structural modal brain network is reconstructed through the decoding stage The goal is to make the reconstructed functional modal brain network similar to the functional modal brain network of the original input, and the reconstructed structural modal brain network similar to the structural modal brain network of the original input, so as to obtain the most representative hidden layer features. Therefore, the loss function of the encoding-decoding stage is: Where N is the number of samples; The specific method of using the self-attention mechanism is: The current feature map is represented by matrix X. After the self-attention calculation, an attention map matrix is ​​obtained. The meaning of each element of the attention map matrix is ​​understood as "weight". Its i-th row element represents the influence of pixels at all positions in the feature map on the i-th pixel. Then, the attention map matrix is ​​multiplied by the input feature map to obtain the feature map weighted by attention, represented by Y. It should be noted that Y is not directly output here, but the idea of ​​residual network is introduced: Y = γ * Y + X, where γ is a parameter, indicating that in this formula, the weight of the feature map after the overall attention is applied needs to be updated through back propagation. X is the original input feature map. In the initial stage, the value of γ is 0, and the input feature map is directly returned. After that, with learning, the feature map weighted by attention is gradually added to the original feature map, thereby emphasizing the part of attention. (6) Comprehensively utilize the latent information of structural mode and functional mode to complete the brain network classification task; The hidden layer of the autoencoder model has a good representation learning ability, which represents the important high-order information in the original input data. In order to comprehensively utilize the information of the two modalities during classification, the hidden layers of the functional modality and the structural modality are spliced ​​together and mapped to a low-dimensional space through a fully connected layer before predicting the label. Since the hidden layer of the structural mode and the hidden layer of the functional mode are distributed in two different spaces, and the information of the two hidden layers is needed to predict the label, it is not reasonable to simply concatenate the two vectors distributed in different spaces. In order to solve this problem, MMD loss is introduced into the model to make the two spaces similar, and the similarity of the two spaces is measured by MMD; the larger the value calculated by MMD loss, the greater the distribution difference of the two spaces, and if the calculated value is small enough, it is considered that the two spaces have the same distribution.

2. The brain network classification method based on community detection and two-way autoencoding according to claim 1, characterized in that: The step (6) comprehensively utilizes the hidden layer information of the structural mode and the functional mode to complete the brain network classification task. In this task, of course, it is hoped that the distribution of the structural mode hidden layer and the distribution of the functional mode hidden layer are as close as possible, so it is hoped to minimize the MMD loss of the two hidden layer spaces, which can be expressed as follows: Here, the Gaussian kernel is selected as the kernel function, so the MMD loss is expressed as: In the classification stage, cross loss entropy is selected as the loss function of the classifier. Therefore, the loss function of the classification stage is defined as Finally, the loss function of the entire model is obtained: Loss sum =Loss rec +αLoss cls +βLoss mmd When training the model, the goal is to minimize the total loss function, learn the best model parameters and hidden layer representation, and then combine the features of the two hidden layers to complete the classification task of the brain network.

3. The brain network classification method based on community detection and two-way autoencoding according to claim 1, characterized in that: By using the method of structural modality to guide functional modality, the community structure in the brain network is detected to adjust the arrangement order of nodes in the brain network. On the one hand, the potential community structure in the network is preserved, and on the other hand, the problem that traditional rectangular convolution kernels are not applicable to non-Euclidean domains such as brain networks is solved. In addition, a two-way autoencoder model with an attention mechanism is used to extract important high-order information from the two modalities, and then the complementary high-order information of the two modalities is comprehensively utilized to complete the classification task of the brain network and improve the classification accuracy.

Images