Brain network analysis method and system based on adaptive graph collaborative contrast learning

Abstract

The invention relates to the technical field of image processing, and provides a brain network analysis method and system based on adaptive graph collaborative contrast learning. The method comprises the following steps: dividing a brain into a plurality of brain regions, extracting an average time sequence of each brain region, and calculating a Pearson's correlation coefficient between every two brain regions; according to the Pearson's correlation coefficient, selecting edges meeting requirements, and constructing a brain map by taking the brain region as a node; performing learnable attribute augmentation on each brain graph by adopting a graph attribute augmentation device to obtain two attribute augmentation views and an attribute augmentation probability matrix; a graph structure augmentation device is adopted to carry out learnable structure augmentation on each brain graph, and two structure augmentation views and a structure augmentation probability matrix are obtained; training a feature extractor, a graph attribute augmentor, a graph structure augmentor and a classifier by utilizing graph collaborative contrast learning; based on the brain map, a trained feature extractor is adopted to extract brain map features, and then a trained classifier is adopted to carry out brain disease classification on the brain map features.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a brain network analysis method and system based on adaptive graph collaborative contrast learning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Resting-state functional magnetic resonance imaging has become a promising auxiliary diagnostic tool in recent years. Resting-state functional magnetic resonance imaging measures low-frequency bold-oxygen-level-dependent (BOLD) signals in the resting state without performing specific tasks. The BOLD signal records the metabolic activity of the brain function of the scanned subject during natural rest, and the correlation of paired BOLD signals can reflect the interaction between the corresponding brain regions. Therefore, the functional connections between multiple brain regions constitute a functional brain network. By studying the functional brain network, we can better understand the pathology of brain diseases.

[0004] With the rise of deep learning, more and more technologies are being developed to analyze functional brain networks and discover more biomarkers. Deep learning models usually require a large number of labeled samples to achieve good generalization performance, while the acquisition and labeling of resting-state functional magnetic resonance imaging data requires a lot of manpower and time costs.

[0005] Graph contrastive learning provides a solution to the above problems. Generally speaking, graph contrastive learning uses data augmentation to generate training views, and then maximizes the similarity between positive samples while minimizing the similarity between negative samples. In this way, graph contrastive learning can learn discriminative embeddings without labels. Traditional graph data augmentation generally adopts predefined augmentation strategies, and selecting the optimal augmentation method requires a lot of trial and error, which takes a lot of time. Summary of the invention

[0006] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a brain network analysis method and system based on adaptive graph collaborative contrastive learning. The present invention uses limited labeled data to train a good feature extractor for brain network analysis, thereby reducing costs and improving efficiency while improving the accuracy of feature extraction for brain disease classification.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A first aspect of the present invention provides a brain network analysis method based on adaptive graph collaborative contrastive learning.

[0009] A brain network analysis method based on adaptive graph collaborative contrastive learning, comprising:

[0010] Based on the acquired brain functional images, the brain is divided into several brain regions; the average time series of each brain region is extracted, and the Pearson correlation coefficient between each two brain regions is calculated; according to the Pearson correlation coefficient, the edges that meet the requirements are selected, the brain regions are regarded as nodes, and the brain map is constructed;

[0011] Using a graph attribute augmentor, each brain map is subjected to learnable attribute augmentation, and two attribute augmentation views and attribute augmentation probability matrices are obtained. Using a graph structure augmentor, each brain map is subjected to learnable structure augmentation, and two structure augmentation views and structure augmentation probability matrices are obtained.

[0012] Based on two attribute augmented views, attribute augmented probability matrix, two structure augmented views and structure augmented probability matrix, feature extractor, graph attribute augmentor, graph structure augmentor and classifier are trained using graph collaborative contrastive learning.

[0013] Based on the brain map, a trained feature extractor is used to extract brain map features, and then a trained classifier is used to classify brain diseases based on the brain map features.

[0014] Furthermore, the graph attribute augmenter is expressed by the following formula:

[0015]

[0016]

[0017]

[0018] in, represents the node embedding of the k-1th layer after node v′ is aggregated by the graph neural network, N(v) represents the set of nodes adjacent to node v, AGGREGATION (k) () represents the aggregation function of the kth layer of the graph neural network, represents the aggregation information of node v at the kth level, represents the node embedding of the k-1th layer after node v is aggregated by the graph neural network, COMBINE(k)() represents the combination function of the kth layer of the graph neural network, represents the node embedding of the kth layer after node v is aggregated by the graph neural network, O v =[o v1 , o v2 , o r3 ] represents the augmented probability of node v obtained by GumbelSoftmax sampling, according to O vThe value of is used to delete the node v, cover the attribute or do not operate it. The same is true for other nodes to obtain the attribute augmented view. The augmented probabilities of all nodes in the graph are arranged together to form an augmented probability matrix P a ∈R n×3 , n represents the number of nodes.

[0019] Furthermore, the graph structure augmentor adopts the following formula:

[0020]

[0021]

[0022]

[0023] p e =Sigmoid((logδ-log(1-δ)+ω e ) / τ), whereδ~Uniform(0,1)

[0024] in, represents the node embedding of the k-1th layer after node v′ is aggregated by the graph neural network, N(v) represents the set of nodes adjacent to node v, AGGREGATION (k) () represents the aggregation function of the kth layer of the graph neural network, represents the aggregation information of node v at the kth level, represents the node embedding of the k-1th layer after node v is aggregated by the graph neural network, COMBINE(k)() represents the combination function of the kth layer of the graph neural network, represents the node embedding of the kth layer after node v is aggregated by the graph neural network, e represents the edge connecting node u and node z, represents the concatenation of the embeddings of node u and node z, MLP represents a multi-layer perceptron, ω e represents the representation of edge e after MLP embedding, δ represents a random number sampled from a uniform distribution, τ represents the temperature parameter, and p e represents the deletion probability of edge e. Finally, some edges are deleted according to the deletion probability of each edge to obtain the structural augmented view. The augmented probabilities of all edges in the graph are arranged together to form the augmented probability matrix P s ∈R m , m represents the number of edges.

[0025] Furthermore, based on two attribute augmented views, an attribute augmented probability matrix, two structure augmented views and a structure augmented probability matrix, graph collaborative contrastive learning is used to train the feature extractor, graph attribute augmentor, graph structure augmentor and classifier; the method includes: using the similarity loss and classification loss of the attribute augmented views of the labeled brain map to optimize the graph attribute augmentor, using the similarity loss and classification loss of the structure augmented views of the labeled brain map to optimize the graph structure augmentor, and optimizing the classifier using the classification loss of the labeled brain map.

[0026] Further, based on the two attribute augmented views, the attribute augmented probability matrix, the two structure augmented views and the structure augmented probability matrix, the feature extractor, the graph attribute augmentor, the graph structure augmentor and the classifier are trained using graph collaborative contrastive learning; the method also includes: using an unlabeled brain map to train the feature extractor, using graph collaborative contrastive learning to optimize the feature extractor, treating the two attribute augmented views as positive sample pairs, and calculating the contrast loss L acl ; Consider the two structure augmented views as positive sample pairs and calculate the contrast loss L scl , randomly select one view from the two attribute augmented views and the two structure augmented views, regard the two selected views as positive sample pairs, and calculate the contrast loss L ccl , the three contrastive losses are added together to perform graph collaborative contrastive learning.

[0027] Furthermore, the method of dividing the brain into several brain regions based on the acquired brain functional images includes: dividing the brain into several brain regions based on an automatic anatomical labeling atlas after preprocessing the brain functional images.

[0028] A second aspect of the present invention provides a brain network analysis system based on adaptive graph collaborative contrastive learning.

[0029] A brain network analysis system based on adaptive graph collaborative contrastive learning, comprising:

[0030] An image processing module is configured to: divide the brain into several brain regions based on the acquired brain functional images; extract the average time series of each brain region, and calculate the Pearson correlation coefficient between two brain regions; select edges that meet the requirements according to the Pearson correlation coefficient, regard the brain regions as nodes, and construct a brain map;

[0031] The augmentation module is configured to: use a graph attribute augmentor to perform learnable attribute augmentation on each brain map, and obtain two attribute augmentation views and an attribute augmentation probability matrix; use a graph structure augmentor to perform learnable structure augmentation on each brain map, and obtain two structure augmentation views and a structure augmentation probability matrix;

[0032] A training optimization module, configured to: train a feature extractor, a graph attribute augmentor, a graph structure augmentor, and a classifier using graph collaborative contrastive learning based on two attribute augmentation views, an attribute augmentation probability matrix, two structure augmentation views, and a structure augmentation probability matrix;

[0033] The classification module is configured to: extract brain map features based on the brain map using a trained feature extractor, and then classify brain diseases based on the brain map features using a trained classifier.

[0034] A third aspect of the present invention provides a computer-readable storage medium.

[0035] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the brain network analysis method based on adaptive graph collaborative contrastive learning as described in the first aspect above.

[0036] A fourth aspect of the present invention provides a computer device.

[0037] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the brain network analysis method based on adaptive graph collaborative contrastive learning as described in the first aspect above are implemented.

[0038] A fifth aspect of the present invention provides a computer device.

[0039] The present invention provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the brain network analysis method based on adaptive graph collaborative contrastive learning as described in the first aspect above.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention provides a brain network analysis method and system based on adaptive graph collaborative contrast learning, the method comprising: preprocessing brain function image data output by a functional magnetic resonance imaging instrument; dividing the preprocessed brain function image into a number of brain regions according to a standard template; calculating the connection weights between the various brain regions based on the Pearson correlation to construct a brain map; performing learnable augmentation on the brain map, using the augmented view to perform graph collaborative contrast learning, and extracting brain network features for analysis. The method simultaneously considers learnable attribute augmentation and structural augmentation, adopts an automatic augmentation method, avoids manual selection and repeated trial and error of graph augmentation strategies, and ensures that the generated view is semantically similar to the original sample. Through graph collaborative contrast learning, the information of the augmented view is fully utilized to train the feature extractor for subsequent brain disease classification, which has important theoretical significance and practical application value.

[0042] The graph augmentation in the present invention is automatic and learnable. The nodes or edges of the graph are modified according to the features of each graph, thereby realizing the augmentation of the graph, avoiding the random augmentation method that may damage the original graph information, and also avoiding the predefined augmentation method of performing the same augmentation operation on all graphs in the data set. By optimizing the graph attribute augmenter and the graph structure augmenter, the graph attribute augmenter and the graph structure augmenter can automatically select the optimal augmentation operation for each graph, avoiding the manual selection and repeated trial and error of the augmentation operation, saving manpower and time costs; and ensuring that the augmented view can retain the semantic information of the original graph, avoiding the labels of the augmented view being different from the original sample, and introducing noise during the training process. After the graph is augmented by learnable augmentation to generate augmented views, graph collaborative comparative learning is performed on multiple augmented views, which includes cross-view comparison, so that the feature extractor can capture rich information from different views, and discriminative embedding can be learned without using labels, alleviating the problem of high cost of functional magnetic resonance imaging data annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0044] Figure 1 It is a flow chart of the brain network analysis method based on adaptive graph collaborative contrastive learning shown in the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0049] Embodiment 1

[0050] The present embodiment provides a brain network analysis method based on adaptive graph collaborative contrast learning. The present embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a terminal, a server, and a system, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In the present embodiment, the method includes the following steps:

[0051] Based on the acquired brain functional images, the brain is divided into several brain regions; the average time series of each brain region is extracted, and the Pearson correlation coefficient between each two brain regions is calculated; according to the Pearson correlation coefficient, the edges that meet the requirements are selected, the brain regions are regarded as nodes, and the brain map is constructed;

[0052] Using a graph attribute augmentor, each brain map is subjected to learnable attribute augmentation, and two attribute augmentation views and attribute augmentation probability matrices are obtained. Using a graph structure augmentor, each brain map is subjected to learnable structure augmentation, and two structure augmentation views and structure augmentation probability matrices are obtained.

[0053] Based on two attribute augmented views, attribute augmented probability matrix, two structure augmented views and structure augmented probability matrix, feature extractor, graph attribute augmentor, graph structure augmentor and classifier are trained using graph collaborative contrastive learning.

[0054] Based on the brain map, a trained feature extractor is used to extract brain map features, and then a trained classifier is used to classify brain diseases based on the brain map features.

[0055] The present invention performs learnable attribute augmentation and structural augmentation on the brain map, performs graph collaborative contrast learning on the augmented views, and trains the feature extractor to capture features from different views. Different from the traditional predefined graph augmentation and graph contrast learning, the present invention performs learnable graph augmentation, performs different augmentations for each graph according to the features of the graph, and the augmenter is continuously optimized during the training process, thereby ensuring that the generated views retain the original semantic information. In addition, the present invention performs cross-view graph contrast learning on the augmented views, so that the feature extractor captures complementary information from different types of views.

[0056] The brain network analysis method based on adaptive graph collaborative contrast learning described in this embodiment is described in detail below with reference to the accompanying drawings. Figure 1 As shown:

[0057] (1) The brain functional images collected by the functional magnetic resonance imaging device were preprocessed using the DPARSF toolbox of Matlab as follows: the first p time points of the magnetic resonance images were removed; the images were corrected for head motion and time layers; the effects of ventricular and white matter signals and high-order effects of head motion were removed; the corrected images were registered to the standard space; and the images were filtered using a 0.01-0.1 Hz time bandpass filter to reduce the effects of heartbeat and breathing.

[0058] Specifically, in this example, a public Autism Brain Imaging Data Exchange (ABIDE) dataset is used, and data from three larger sites are selected, namely New York University (NYU), University of Michigan (UM) and University of Leuven (LEUVEN). There are 184 subjects at the NYU site, including 79 autistic patients and 105 normal people; there are 145 subjects at the UM site, including 68 autistic patients and 77 normal people; there are 64 subjects at the LEUVEN site, including 29 autistic patients and 35 normal people. The format of the brain images collected by the magnetic resonance device is read and converted. In order to keep the time series stable, the first 10 time points of each subject's magnetic resonance image are removed; the image is then corrected for head motion and time layer; the influence caused by ventricular, white matter signals and head motion high-order effects is removed; the corrected image is registered to the standard space; the subject image is filtered with a 0.01-0.1HZ time bandpass filter to reduce the influence of heartbeat and breathing.

[0059] (2) After preprocessing the brain functional image data, the brain is divided into several brain regions based on the automatic anatomical labeling atlas, and the average time series of each brain region is extracted and expressed as S = [s1, s2, ..., s i ,…,s n ]∈R t×n , where s i ∈R t represents the time series of the ith brain region, t represents the length of the time series, and n represents the number of brain regions.

[0060] Specifically, a standardized brain partition template is selected. In this example, the human brain is divided into 116 brain regions (90 brain regions of the cerebrum and 26 brain regions of the cerebellum) according to the AAL template, and the 116 brain regions respectively represent 116 nodes of the brain functional connection network.

[0061] (3) Based on the average time series of brain regions, the Pearson correlation coefficient between each pair of brain regions was calculated, expressed as W = [w 11 , w 12 ,…,w ij ,…,w nn ]∈R n×n , w ij represents the correlation between the i-th brain region and the j-th brain region.

[0062] (4) Considering brain regions as nodes, using the Pearson correlation coefficient between brain regions, we select the top 30% of brain region pairs according to the strength of correlation, connect them with edges, and construct a brain map, represented by G = (X, A), where X = W∈R n×n , represents the node feature matrix, A=[a 11 , a 12 , …, a ij,…,a nn ]∈R n×n Represents the adjacency matrix of the graph, a ij =1 means there is an edge between the i-th node and the j-th node, otherwise, a ij =0 means there is no edge between the i-th node and the j-th node.

[0063] (5) Perform learnable attribute augmentation on each brain map and create two graph attribute augmenters, each generating an augmented view, thus obtaining two attribute augmented views. The graph attribute augmenters are as follows:

[0064]

[0065]

[0066]

[0067] in, represents the node embedding of the k-1th layer after node v′ is aggregated by the graph neural network, N(v) represents the set of nodes adjacent to node v, AGGREGATION (k) () represents the aggregation function of the kth layer of the graph neural network, represents the aggregation information of node v at the kth level, represents the node embedding of the k-1th layer after node v is aggregated by the graph neural network, COMBINE(k)() represents the combination function of the kth layer of the graph neural network, represents the node embedding of the kth layer after node v is aggregated by the graph neural network, O v =[o v1 , o v2 , o v3 ] represents the augmented probability of node v obtained by GumbelSoftmax sampling, according to O v The value of is used to delete the node v, cover the attribute or do not operate it. The same is true for other nodes to obtain the attribute augmented view. The augmented probabilities of all nodes in the graph are arranged together to form the augmented probability matrix P a ∈R n×3 , n represents the number of nodes.

[0068] (6) Perform learnable structural augmentation on each brain map and create two graph structure augmenters, each generating an augmented view, to obtain two structure augmented views. The graph structure augmenters are as follows:

[0069]

[0070]

[0071]

[0072] p e =Sigmoid((logδ-log(1-δ)+ω e ) / τ), whereδ~Uniform(0,1)

[0073] in, represents the node embedding of the k-1th layer after node v′ is aggregated by the graph neural network, N(v) represents the set of nodes adjacent to node v, AGGREGATION (k) () represents the aggregation function of the kth layer of the graph neural network, represents the aggregation information of node v at the kth level, represents the node embedding of the k-1th layer after node v is aggregated by the graph neural network, COMBINE(k)() represents the combination function of the kth layer of the graph neural network, represents the node embedding of the kth layer after node v is aggregated by the graph neural network, e represents the edge connecting node u and node z, represents the concatenation of the embeddings of node u and node z, MLP represents a multi-layer perceptron, ω e represents the representation of edge e after MLP embedding, δ represents a random number sampled from a uniform distribution, τ represents the temperature parameter, and p e represents the deletion probability of edge e. Finally, some edges are deleted according to the deletion probability of each edge to obtain the structural augmented view. The augmented probabilities of all edges in the graph are arranged together to form the augmented probability matrix P s ∈R m , m represents the number of edges.

[0074] (7) For each brain map’s four augmented views, we use graph collaborative contrastive learning for training. The training process is divided into two parts:

[0075] First: Using labeled data, we use the similarity loss and classification loss of the attribute augmented view to optimize the graph attribute augmentor, use the similarity loss and classification loss of the structure augmented view to optimize the graph structure augmentor, and use the classification loss of the original graph to optimize the classifier. When optimizing the graph augmentor, the similarity loss is used to reduce the similarity between the two views, and the classification loss is used to ensure that the label of the augmented view is the same as the original sample.

[0076] The similarity loss is defined as:

[0077]

[0078] Among them, P i1 and P i2 represents the augmented probability matrix of the two attribute augmented views (or structure augmented views) of the i-th sample, |P i1 | represents the matrix Pi1 The number of elements in , N represents the number of samples.

[0079] The classification loss used to update the graph augmentor is defined as:

[0080]

[0081]

[0082]

[0083] Among them, l cls () represents the cross entropy loss function, y i represents the true label of the i-th sample, and Represents the predicted labels of the two attribute augmented views (or structure augmented views) of the i-th sample after the feature extractor and classifier, and N represents the number of samples.

[0084] The total loss for updating the graph augmentor is defined as:

[0085] L joint =λ·L sim +L cls_aug

[0086] Wherein, λ∈[0,1] represents a compromise parameter, which is set to 0.8 in this embodiment.

[0087] The classification loss used to update the classifier is defined as:

[0088]

[0089]

[0090] Among them, l cls () represents the cross entropy loss function, y i represents the true label of the i-th sample, It represents the predicted label of the i-th sample after the feature extractor and classifier, and N represents the number of samples.

[0091] Second: Using unlabeled data, we use graph collaborative contrastive learning to optimize the GNN feature extractor, treating the two attribute augmented views as positive sample pairs and calculating the contrast loss L acl ; Consider the two structure augmented views as positive sample pairs and calculate the contrast loss L scl In addition, a view is randomly selected from the two attribute augmented views and the two structure augmented views, and the two selected views are regarded as positive sample pairs to calculate the contrast loss L ccl , the three contrastive losses are added together to perform graph collaborative contrastive learning.

[0092] L acl Defined as:

[0093]

[0094]

[0095]

[0096]

[0097] Among them, v i1 and v i2 represents the two attribute augmented views of the i-th graph, z i1 and z i2 Indicates v i1 and v i2 The embedding obtained by the feature extractor, sim() represents the similarity function, calculated using cosine similarity, τ is the temperature parameter, and N represents the number of samples.

[0098] L scl Defined as:

[0099]

[0100]

[0101]

[0102] Among them, v i3 and v i4 represents the two structural augmented views of the i-th graph, z i3 and z i4 Indicates v i3 and v i4 The embedding obtained by the feature extractor, sim() represents the similarity function, which is similar to L acl The similarity function in uses the same calculation method, τ is the temperature parameter, and N represents the number of samples.

[0103] L ccl Defined as:

[0104]

[0105]

[0106]

[0107] Among them, v i5 represents a view randomly selected from the two attribute augmented views of the i-th graph, v i6represents a view randomly selected from the two structured augmented views of the i-th graph, z i5 and z i6 Indicates v i5 and v i6 The embedding obtained by the feature extractor, sim() represents the similarity function, which is similar to L acl The similarity function in uses the same calculation method, τ is the temperature parameter, and N represents the number of samples.

[0108] The total contrast loss is:

[0109]

[0110] (8) Repeat the above training process until the number of iterations reaches the set value H, stop updating, and obtain a trained feature extractor for brain disease classification.

[0111] (9) Steps (5)(6)(7)(8) are performed only on the NYU site, and a feature extractor that is well trained on the NYU site is obtained. In order to verify the effectiveness of the proposed method, this embodiment transfers the parameters of the feature extractor trained on the NYU site to the autism patient vs. normal person recognition experiment at the UM site and the LEUVEN site, and fine-tunes the feature extractor and classifier at the UM site and the LEUVEN site, respectively. Further explanation: The feature extractor used in this embodiment is a graph isomorphism network (Graph Isomorphism Network, GIN). In order to avoid the influence of complex classifiers on the experimental results, this embodiment uses a two-layer multi-layer perceptron (Multi-Layer Perceptron, MLP) as a classifier. Model effectiveness evaluation indicators include accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curve (The area under the receiver operating characteristic curve, AUC). The definitions of the indicators are: Accuracy = (TP+TN) / (TP+FN+FP+TN), Sensitivity = TP / (TP+FN), Specificity = TM / (TM+TP), where TP, TN, FP and FN represent true positive, true negative, false positive and false negative, respectively.

[0112] (10) As shown in Table 1, the classification index performance of the five baseline methods, Window Slice, Time Warp, Node Dropping, BrainGB, and NoPretrain, and the adaptive graph collaborative contrastive learning model proposed in the present invention are shown. Among them, the present invention has achieved the best performance in all indicators. On the UM site, Accuracy = 70.34%, Sensitivity = 67.58%, Specificity = 72.75%, AUC = 69.71%; on the LEUVEN site, Accuracy = 70.77%, Sensitivity = 63.33%, Specificity = 77.14%, AUC = 71.90%. It is found that the classification performance of Window Slice is poor. The possible reason is that Window Slice performs data augmentation by window slicing the time series, which may damage the information of the original BOLD signal, thereby affecting feature extraction.

[0113] Table 1 Classification index performance of adaptive graph collaborative contrastive learning model

[0114]

[0115]

[0116] Embodiment 2

[0117] This embodiment provides a brain network analysis system based on adaptive graph collaborative contrastive learning.

[0118] A brain network analysis system based on adaptive graph collaborative contrastive learning, comprising:

[0119] An image processing module is configured to: divide the brain into several brain regions based on the acquired brain functional images; extract the average time series of each brain region, and calculate the Pearson correlation coefficient between two brain regions; select edges that meet the requirements according to the Pearson correlation coefficient, regard the brain regions as nodes, and construct a brain map;

[0120] The augmentation module is configured to: use a graph attribute augmentor to perform learnable attribute augmentation on each brain map, and obtain two attribute augmentation views and an attribute augmentation probability matrix; use a graph structure augmentor to perform learnable structure augmentation on each brain map, and obtain two structure augmentation views and a structure augmentation probability matrix;

[0121] A training optimization module, configured to: train a feature extractor, a graph attribute augmentor, a graph structure augmentor, and a classifier using graph collaborative contrastive learning based on two attribute augmentation views, an attribute augmentation probability matrix, two structure augmentation views, and a structure augmentation probability matrix;

[0122] The classification module is configured to: extract brain map features based on the brain map using a trained feature extractor, and then classify brain diseases based on the brain map features using a trained classifier.

[0123] In some embodiments, the augmentation module is further configured as follows: the graph attribute augmentor is expressed by the following formula:

[0124]

[0125]

[0126]

[0127] in, represents the node embedding of the k-1th layer after node v′ is aggregated by the graph neural network, N(v) represents the set of nodes adjacent to node v, AGGREGATION (k) () represents the aggregation function of the kth layer of the graph neural network, represents the aggregation information of node v at the kth level, represents the node embedding of the k-1th layer after node v is aggregated by the graph neural network, COMBINE (k) () represents the combination function of the kth layer of the graph neural network, represents the node embedding of the kth layer after node v is aggregated by the graph neural network, O v =[o v1 ,o v2 ,o v3 ] represents the augmented probability of node v obtained by GumbelSoftmax sampling, according to O v The value of is used to delete the node v, cover the attribute or do not operate it. The same is true for other nodes to obtain the attribute augmented view. The augmented probabilities of all nodes in the graph are arranged together to form an augmented probability matrix P a ∈R n×3 , n represents the number of nodes.

[0128] In some embodiments, the augmentation module is further configured as follows: the graph structure augmentor adopts the following formula:

[0129]

[0130]

[0131]

[0132] p e =Sigmoid((logδ-log(1-δ)+ω e) / τ),whereδ~Uniform(0,1)

[0133] in, represents the node embedding of the k-1th layer after node v′ is aggregated by the graph neural network, N(v) represents the set of nodes adjacent to node v, AGGREGATION (k) () represents the aggregation function of the kth layer of the graph neural network, represents the aggregation information of node v at the kth level, represents the node embedding of the k-1th layer after node v is aggregated by the graph neural network, COMBINE (k) () represents the combination function of the kth layer of the graph neural network, represents the node embedding of the kth layer after node v is aggregated by the graph neural network, e represents the edge connecting node u and node z, represents the concatenation of the embeddings of node u and node z, MLP represents a multi-layer perceptron, ω e represents the representation of edge e after MLP embedding, δ represents a random number sampled from a uniform distribution, τ represents the temperature parameter, and p e represents the deletion probability of edge e. Finally, some edges are deleted according to the deletion probability of each edge to obtain the structural augmented view. The augmented probabilities of all edges in the graph are arranged together to form the augmented probability matrix P s ∈R m , m represents the number of edges.

[0134] In some embodiments, the training optimization module is further configured to: optimize the graph attribute augmentor using the similarity loss and classification loss of the attribute augmented view of the labeled brain map, optimize the graph structure augmentor using the similarity loss and classification loss of the structure augmented view of the labeled brain map, and optimize the classifier using the classification loss of the labeled brain map. Train the feature extractor using the unlabeled brain map, optimize the feature extractor using graph collaborative contrastive learning, regard the two attribute augmented views as positive sample pairs, and calculate the contrast loss L acl ; Consider the two structure augmented views as positive sample pairs and calculate the contrast loss L scl , randomly select one view from the two attribute augmented views and the two structure augmented views, regard the two selected views as positive sample pairs, and calculate the contrast loss L ccl , the three contrastive losses are added together to perform graph collaborative contrastive learning.

[0135] In some embodiments, the image processing module is further configured to: after pre-processing the brain functional image, divide the brain into several brain regions based on the automatic anatomical labeling atlas.

[0136] Embodiment 3

[0137] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the brain network analysis method based on adaptive graph collaborative contrastive learning as described in the first embodiment above are implemented.

[0138] Embodiment 4

[0139] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the brain network analysis method based on adaptive graph collaborative contrastive learning as described in the first embodiment are implemented.

[0140] Embodiment 5

[0141] This embodiment provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned brain network analysis method based on adaptive graph collaborative contrastive learning.

[0142] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0143] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A brain network analysis method based on adaptive graph collaborative contrastive learning, characterized in that: include: Based on the acquired brain functional images, the brain is divided into several brain regions; the average time series of each brain region is extracted, and the Pearson correlation coefficient between two brain regions is calculated; According to the Pearson correlation coefficient, edges that meet the requirements are selected, and brain regions are regarded as nodes to construct a brain map; Using a graph attribute augmentor, each brain map is subjected to learnable attribute augmentation, and two attribute augmentation views and an attribute augmentation probability matrix are obtained. Using a graph structure augmentor, each brain map is subjected to learnable structure augmentation, and two structure augmentation views and a structure augmentation probability matrix are obtained. Based on two attribute augmented views, attribute augmented probability matrix, two structure augmented views and structure augmented probability matrix, feature extractor, graph attribute augmentor, graph structure augmentor and classifier are trained using graph collaborative contrastive learning. Based on the brain map, a trained feature extractor is used to extract brain map features, and then a trained classifier is used to classify brain diseases based on the brain map features.

2. The brain network analysis method based on adaptive graph collaborative contrastive learning according to claim 1 is characterized in that: The graph attribute augmenter is expressed by the following formula: in, represents the node embedding of the k-1th layer after node v′ is aggregated by the graph neural network, N(v) represents the set of nodes adjacent to node v, AGGREGATION (k) () represents the aggregation function of the kth layer of the graph neural network, represents the aggregation information of node v at the kth level, represents the node embedding of the k-1th layer after node v is aggregated by the graph neural network, COMBINE (k) () represents the combination function of the kth layer of the graph neural network, represents the node embedding of the kth layer after node v is aggregated by the graph neural network, O v =[o v1 ,o v2 ,o v3 ] represents the augmented probability of node v obtained by GumbelSoftmax sampling, according to O v The value of is used to delete the node v, cover the attribute or do not operate it. The same is true for other nodes to obtain the attribute augmented view. The augmented probabilities of all nodes in the graph are arranged together to form an augmented probability matrix P a ∈R n×3 , n represents the number of nodes.

3. The brain network analysis method based on adaptive graph collaborative contrastive learning according to claim 1 is characterized in that: The graph structure augmenter adopts the following formula: p e =Sigmoid((logδ-log(1-δ)+ω e ) / τ),whereδ~Uniform(0,1) in, Represents node v ′ The node embedding of the k-1th layer after aggregation by the graph neural network, N(v) represents the set of nodes adjacent to node v, AGGREGATION (k) () represents the aggregation function of the kth layer of the graph neural network, represents the aggregation information of node v at the kth level, represents the node embedding of the k-1th layer after node v is aggregated by the graph neural network, COMBINE (k) () represents the combination function of the kth layer of the graph neural network, represents the node embedding of the kth layer after node v is aggregated by the graph neural network, e represents the edge connecting node u and node z, represents the concatenation of the embeddings of node u and node z, MLP represents a multi-layer perceptron, ω e represents the representation of edge e after MLP embedding, δ represents a random number sampled from a uniform distribution, τ represents the temperature parameter, and p e represents the deletion probability of edge e. Finally, some edges are deleted according to the deletion probability of each edge to obtain the structural augmented view. The augmented probabilities of all edges in the graph are arranged together to form the augmented probability matrix P s ∈R m , m represents the number of edges.

4. The brain network analysis method based on adaptive graph collaborative contrastive learning according to claim 1, characterized in that: Based on two attribute augmented views, an attribute augmented probability matrix, two structure augmented views and a structure augmented probability matrix, graph collaborative contrastive learning is used to train a feature extractor, a graph attribute augmentor, a graph structure augmentor and a classifier; the method includes: using the similarity loss and classification loss of the attribute augmented views of labeled brain maps to optimize the graph attribute augmentor, using the similarity loss and classification loss of the structure augmented views of labeled brain maps to optimize the graph structure augmentor, and optimizing the classifier using the classification loss of labeled brain maps.

5. The brain network analysis method based on adaptive graph collaborative contrastive learning according to claim 4 is characterized in that: Based on two attribute augmented views, an attribute augmented probability matrix, two structure augmented views and a structure augmented probability matrix, a feature extractor, a graph attribute augmentor, a graph structure augmentor and a classifier are trained using graph collaborative contrastive learning; the method also includes: using an unlabeled brain map to train the feature extractor, using graph collaborative contrastive learning to optimize the feature extractor, treating the two attribute augmented views as positive sample pairs, and calculating the contrast loss L acl ; Consider the two structure augmented views as positive sample pairs and calculate the contrast loss L scl , randomly select one view from the two attribute augmented views and the two structure augmented views, regard the two selected views as positive sample pairs, and calculate the contrast loss L ccl , the three contrastive losses are added together to perform graph collaborative contrastive learning.

6. The brain network analysis method based on adaptive graph collaborative contrastive learning according to claim 1, characterized in that: The method of dividing the brain into several brain regions based on the acquired brain function images includes: dividing the brain into several brain regions based on the automatic anatomical labeling atlas after preprocessing the brain function images.

7. A brain network analysis system based on adaptive graph collaborative contrastive learning, characterized in that: include: An image processing module is configured to: divide the brain into several brain regions based on the acquired brain functional images; extract the average time series of each brain region, and calculate the Pearson correlation coefficient between two brain regions; According to the Pearson correlation coefficient, edges that meet the requirements are selected, and brain regions are regarded as nodes to construct a brain map; An augmentation module is configured to: use a graph attribute augmentor to perform learnable attribute augmentation on each brain map, and obtain two attribute augmentation views and an attribute augmentation probability matrix; Using a graph structure augmentor, each brain map is subjected to learnable structure augmentation, and two structure augmentation views and a structure augmentation probability matrix are obtained. A training optimization module, configured to: train a feature extractor, a graph attribute augmentor, a graph structure augmentor, and a classifier using graph collaborative contrastive learning based on two attribute augmentation views, an attribute augmentation probability matrix, two structure augmentation views, and a structure augmentation probability matrix; The classification module is configured to: extract brain map features based on the brain map using a trained feature extractor, and then classify brain diseases based on the brain map features using a trained classifier.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the brain network analysis method based on adaptive graph collaborative contrastive learning as described in any one of claims 1 to 6 are implemented.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the brain network analysis method based on adaptive graph collaborative contrastive learning as described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps in the brain network analysis method based on adaptive graph collaborative contrastive learning as described in any one of claims 1 to 6 are implemented.

Images