Functional brain network construction method and device, medium and equipment

Abstract

The invention discloses a functional brain network construction method and device, a medium and equipment, and relates to the technical field of functional brain network construction. The method comprises the following steps: firstly, based on resting-state brain function magnetic resonance imaging of a patient with potential brain diseases, determining time sequence characteristics of each brain region according to brain region division of an automatic anatomical marker map, constructing an initial function brain network, and then performing spatial convolution on the time sequence characteristics of each brain region to obtain a brain function network; fusing the time sequence features of each brain region after spatial convolution with information of adjacent brain regions in the initial functional brain network, clustering the time sequence features of each brain region after spatial convolution to obtain a clustering mapping matrix, and performing clustering mapping to obtain the time sequence features of each brain region after brain region clustering; and finally, constructing a functional brain network after brain region clustering according to the time sequence features of the clustered brain regions. The functional brain network constructed by the method can more directly characterize potential brain disease characteristics.

Description

Technical Field

[0001] The present invention relates to the technical field of functional brain network construction, and in particular to a method, device, medium and equipment for constructing a functional brain network. Background Art

[0002] Currently, autism spectrum disorder is a common neurodevelopmental disease in children, with main characteristics including impaired social skills, language communication barriers, narrow interests, and repetitive and stereotyped behaviors. Currently, the diagnosis of autism mostly relies on the observation of clinical symptoms and the subjective experience of doctors, which may lead to a high misdiagnosis rate and delay treatment. Therefore, how to assist doctors in diagnosing autism through brain science, especially in the early stages of the disease, has become a research focus in the field of brain science.

[0003] As a non-invasive technology, resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used in the early diagnosis of brain diseases. Taking autism as an example, since autism spectrum disorders usually affect the neural connections in different areas of the brain, building high-quality brain networks has become an important means to assist in identifying abnormalities in the brain areas of autistic patients.

[0004] However, when traditional methods construct brain networks based on resting-state functional magnetic resonance imaging of patients with potential brain diseases, they are usually constructed based on traditional brain regions. Traditional brain region classification is too rigid, and the relationships between a large number of fine-grained brain regions are intricate, which is not conducive to direct observation and judgment by doctors. Therefore, how to construct a more relevant brain network that can more directly characterize potential brain diseases is an urgent problem that needs to be solved. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, medium and equipment for constructing a functional brain network to address the above-mentioned technical problems.

[0006] The present invention adopts the following technical solutions:

[0007] The present invention provides a method for constructing a functional brain network, comprising:

[0008] Based on the resting-state functional magnetic resonance imaging of patients with potential brain diseases, the time series characteristics of each brain region are determined according to the brain region division of the automatic anatomical labeling atlas, and the initial functional brain network is constructed;

[0009] According to the spatial relationship between the nodes in the initial functional brain network, the time series features of each brain region are spatially convolved, so that the time series features of each brain region after spatial convolution are integrated with the information of its adjacent nodes;

[0010] Clustering the time series features of each brain region after spatial convolution to eliminate brain regions that are not related to potential brain diseases and aggregate brain regions with similar time series features after spatial convolution to determine the cluster mapping matrix;

[0011] The time series characteristics of each brain region after spatial convolution are clustered and mapped according to the cluster mapping matrix, and the time series characteristics of each brain region after cluster mapping are determined to construct a functional brain network after brain region clustering.

[0012] Optionally, performing spatial convolution on the time series features of each brain region according to the spatial relationship between each node in the initial functional brain network specifically includes:

[0013] The time series features of each brain region and the initial functional brain network are input into the graph isomorphic network, and the time series features of each brain region are spatially convolved according to the spatial relationship between the nodes in the initial functional brain network using the following formula:

[0014]

[0015] H 0 =AX,

[0016]

[0017] Among them, Z is the time series characteristics of each brain region after spatial convolution, A is the initial functional brain network, X is the time series characteristics of each brain region, GIN_C( ) is the graph isomorphism network expression, is the characteristic information of the input node in the kth layer of the graph isomorphic network, N i is the node adjacent to the i-th node in the initial functional brain network, conv2d() is the convolution function, l is the preset maximum number of iterations of the graph isomorphic network, ∈ k are learnable parameters.

[0018] Optionally, clustering the time series features of each brain region after spatial convolution specifically includes:

[0019] S = softmax(GIN_C p (A,Z)),

[0020]

[0021] Among them, S is the clustering mapping matrix, GIN_C p ( ) is a graph isomorphic network whose readout function is the pooling layer, and pool( ) is the pooling layer function.

[0022] Optionally, the method further comprises:

[0023] The temporal convolution network is used to perform temporal convolution on the time series features of each brain region after brain region clustering through the following formula:

[0024] Tconv(Z')=relu(bn(conv2d'(Z')));

[0025] The time series features of each brain region after time convolution are convolved in time and space by the following formula:

[0026] STGconv(Z')=Tconv(GIN_C(A',Z'))+bn(Tconv(Z'));

[0027] The Transformer model is used to integrate the characteristic information of a single brain region in the time dimension according to the time series characteristics of each brain region after time convolution and the time series characteristics of each brain region after time-space convolution through the following formula, and the coarse-grained time series characteristics of each brain region after time-space aggregation are obtained:

[0028] Q=W q (Tconv(Z' T )),

[0029] K=W k (STGconv(Z' T )),

[0030] V=W v (Tconv(Z' T )),

[0031]

[0032] Where Tconv( ) represents temporal convolution, conv2d'( ) is the convolution function with a kernel size of (1, kernal_size), kernal_size is the size of the kernel in the time dimension, Z' is the time series feature of each brain region after brain region clustering, bn( ) is the batch normalization function, relu( ) is the activation function, STGconv( ) represents spatiotemporal convolution, A' is the functional brain network after brain region clustering, Q is the query constructed by the Transformer model, and W q ( ) is the query constructor of the Transformer model, K is the key constructed by the Transformer model, and W k ( ) is the key constructor of the Transformer model, V is the value constructed by the Transformer model, and W v ( ) is the value constructor of the Transformer model, V out is the time series feature of each brain region after spatiotemporal aggregation, and n' is the number of brain regions after brain region clustering.

[0033] Optionally, the method is based on resting-state functional magnetic resonance imaging of patients with potential brain diseases, determining the time series characteristics of each brain region according to the brain region division of the automatic anatomical labeling atlas, and constructing an initial functional brain network, specifically including:

[0034] Performing slice time correction and head motion correction on the user's resting-state brain functional magnetic resonance image, and removing the influence of the ventricle, white matter signals and the high-order effect of head motion from the corrected resting-state brain functional magnetic resonance image, to obtain a resting-state brain functional magnetic resonance image after preliminary preprocessing;

[0035] The resting-state functional magnetic resonance images after preliminary preprocessing are registered to the standard space, and the registered resting-state functional magnetic resonance images are subjected to 0.01-0.1 Hz time bandpass filtering to reduce the influence of heartbeat and breathing, so as to obtain the final resting-state functional magnetic resonance images after preprocessing;

[0036] According to the automatic anatomical labeling atlas, the brain is divided into multiple brain regions, and the average time series characteristics of each brain region are extracted based on the final preprocessed resting-state functional magnetic resonance images as the time series characteristics of each brain region.

[0037] The present invention provides a functional brain network construction device, comprising:

[0038] An acquisition module is used to determine the time series characteristics of each brain region based on the resting-state functional magnetic resonance imaging of patients with potential brain diseases according to the brain region division of the automatic anatomical labeling atlas, and to construct an initial functional brain network;

[0039] The spatial convolution module is used to perform spatial convolution on the time series features of each brain region according to the spatial relationship between the nodes in the initial functional brain network, so that the time series features of each brain region after spatial convolution are integrated with the information of its adjacent nodes;

[0040] A clustering module is used to cluster the time series features of each brain region after spatial convolution, so as to eliminate brain regions that are not related to potential brain diseases and aggregate brain regions with similar time series features after spatial convolution to determine the cluster mapping matrix;

[0041] The construction module is used to cluster and map the time series characteristics of each brain region after spatial convolution according to the cluster mapping matrix, determine the time series characteristics of each brain region after cluster mapping to construct a functional brain network after brain region clustering.

[0042] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned functional brain network construction method is implemented.

[0043] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned functional brain network construction method when executing the program.

[0044] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0045] The present invention first obtains the user's resting-state brain functional magnetic resonance imaging, determines the time series characteristics of the brain regions according to the traditional brain region division, and constructs an initial functional brain network based on this, then performs spatial convolution on the time series characteristics of each brain region according to the spatial relationship between the nodes in the initial functional brain network, so that the time series characteristics of each brain region after the spatial convolution are integrated with the information of its adjacent brain regions in the initial functional brain network, and then clusters the time series characteristics of each brain region after the spatial convolution to eliminate brain regions that are not related to potential brain diseases and aggregate brain regions with similar time series characteristics after the spatial convolution, thereby determining a cluster mapping matrix, and cluster mapping is performed on the time series characteristics of each brain region after the spatial convolution according to the cluster mapping matrix, and the time series characteristics of each brain region after the cluster mapping are determined to construct a functional brain network after brain region clustering.

[0046] The traditional brain region division has a high granularity, which leads to the complicated relationship between a large number of fine-grained brain regions, which is not conducive to direct observation and judgment. In general, not all brain regions are related to potential brain diseases. Therefore, on the one hand, the present invention aggregates traditional brain regions and eliminates brain regions that are not related to potential brain diseases during the aggregation mapping process, thereby improving the direct correlation between the functional brain network constructed thereby and potential brain diseases, and avoiding the interference of irrelevant brain regions in the characterization of potential brain diseases. On the other hand, the function of the brain region is characterized by its time series characteristics, and brain regions with similar functions are aggregated. By reducing the number of brain regions, the relationship between brain regions is clarified and centralized, thereby reducing the complexity of the functional brain network constructed thereby. The combination of these two aspects improves the degree of representation of the characteristics of potential brain diseases by the constructed functional brain network. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0048] Figure 1 A schematic diagram of a functional brain network construction method provided by the present invention;

[0049] Figure 2 A schematic diagram of a specific process of a method for constructing a functional brain network provided by the present invention;

[0050] Figure 3 A schematic diagram of a functional brain network construction device provided by the present invention;

[0051] Figure 4 A schematic diagram of a computer device for implementing a method for constructing a functional brain network provided by the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0053] At present, the region of interest (ROI) obtained by resting-state functional magnetic resonance imaging preprocessing is represented by the blood oxygen level-dependent (BOLD) signal, which consists of time series. When using classic graph neural networks (GNNs), the fully connected layer as a readout function often destroys the order of the time series while capturing the relationship between adjacent time series, so it is suitable for processing BOLD signals.

[0054] When constructing a brain network, the time series of the brain region itself can be used as a feature, recorded as a node, and the relationship between brain regions can be used as an edge, which is very similar to a graph structure. Therefore, when processing the spatial relationship between brain regions, GNN is often used for operation. The advantage of the graph is that it can mathematically model the unconventional interactions between data based on nodes and edges. In the classical graph theory method, the brain network is analyzed by relying on the intrinsic properties of the graph, such as local efficiency, global efficiency, average path length, and small world. These features can be used to compare brain networks of different subjects, reveal the properties of brain networks, and thus provide insights into biological indicators of brain diseases. The classification results of the Graph Isomorphism Network (GIN) in the Weisfeiler-Lehman (WL) graph isomorphism test neural network prove that the GIN method can provide more powerful classification performance. GIN is suitable for processing data with weak contextual relationships, but the BOLD signal is a time series with strong correlation and is not suitable for GIN. Therefore, the present invention improves GIN and uses two-dimensional convolution as the output layer of GIN. The two-dimensional convolution has a very strong local information aggregation ability and does not destroy the original time characteristics of the sequence.

[0055] The Transformer model is mainly used to process sequence data, and it mainly captures the relationship between different positions in the sequence through the self-attention system. The classic Transformer model is suitable for processing text data, but it is not satisfactory when processing brain area data. In order to meet the characteristics of time series, the present invention uses space-time convolution and time convolution to construct the query Q, key K, and value V of the Transformer model. The space-time convolution here is a novel method that mainly focuses on the time dimension, and the space dimension plays an auxiliary role.

[0056] Furthermore, the main idea of ​​the present invention is the hierarchical spatiotemporal multi-granularity, which uses spatial convolution to aggregate the time dimension, constructs coarse-grained time information, and aggregates fine-grained brain regions to form coarse-grained brain regions. The characteristics of the relationship coefficients between coarse-grained brain regions are more significant, which is convenient for assisting doctors to identify whether the brain network of the subject has the possibility of potential brain diseases.

[0057] The technical solutions provided by various embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0058] Figure 1 The figure is a flow chart of a method for constructing a functional brain network in the present invention, which specifically includes the following steps:

[0059] S101: Based on resting-state functional magnetic resonance imaging of patients with potential brain diseases, the time series characteristics of each brain region are determined according to the brain region division of the automatic anatomical labeling atlas, and an initial functional brain network is constructed.

[0060] S102: performing spatial convolution on the time series features of each brain region according to the spatial relationship between the brain regions in the initial functional brain network, so that the time series features of each brain region after the spatial convolution are integrated with the information of its adjacent brain regions.

[0061] S103: Clustering the time series features of each brain region after spatial convolution to eliminate brain regions that are not related to potential brain diseases and aggregate brain regions with similar time series features after spatial convolution to determine a cluster mapping matrix.

[0062] S104: performing cluster mapping on the time series features of each brain region after spatial convolution according to the cluster mapping matrix, and determining the time series features of each brain region after cluster mapping to construct a functional brain network after brain region clustering.

[0063] For the convenience of explanation, the following description is only based on the server as the execution subject. The server mentioned in the present invention can be a server set up on the business platform, or a device such as a desktop computer, a notebook computer, etc. that can execute the solution of the present invention.

[0064] In practical applications, the server of the service platform can usually first obtain the user's resting state brain functional magnetic resonance imaging and use it to construct the user's functional brain network. Of course, in one or more embodiments of the present invention, the server can first pre-process the user's resting state brain functional magnetic resonance imaging through the DPARSF toolbox of matlab.

[0065] Specifically, the server can first perform slice time correction (removing the first p time points of the resting-state functional MRI image) and head motion correction on the user's resting-state functional MRI image, and remove the influence of ventricle, white matter signals and high-order effects of head motion in the corrected resting-state functional MRI image, to obtain the resting-state functional MRI image after preliminary preprocessing.

[0066] Then, the resting-state functional magnetic resonance images after preliminary preprocessing are registered to the standard space, and the registered resting-state functional magnetic resonance images are filtered through a 0.01-0.1HZ time bandpass filter to reduce the influence of heartbeat and breathing, so as to obtain the final preprocessed resting-state functional magnetic resonance images. Among them, the so-called registration to the standard space means that the magnetic resonance image is usually a three-dimensional image with a time axis (length, width, height, time). Since different brains collected by different machines have different positions in three-dimensional space, it is necessary to use special tools (such as FSL, SPM, AFNI, etc.) to detect and correct head movement to ensure that all resting-state functional magnetic resonance images are in the same coordinate system.

[0067] Finally, the brain is divided into multiple brain regions according to the automatic anatomical labeling atlas, and the average time series features of each brain region are extracted according to the final preprocessed resting-state functional magnetic resonance images as the time series features of each brain region. The time series features of each brain region can be expressed as X = [x1, x2, …, x i ,…,x n ]∈R n×t Among them, x i ∈R n×t represents the time series of the ith brain region, t represents the length of the time series, and n represents the number of brain regions.

[0068] After obtaining the time series characteristics of each brain region, the server can use Pearson correlation to construct an initial functional brain network. Here, the server can determine the correlation between each brain region through the Pearson correlation coefficient based on the time series characteristics of each brain region, so as to use each brain region as a node and determine the edges between each node based on the correlation between each brain region. For example, the Pearson correlation coefficient between brain regions can be used as the edge between nodes. It can also be based on a preset correlation threshold. When the Pearson correlation coefficient between brain regions is greater than the preset correlation threshold, it is considered that there is an edge connection between the nodes of the corresponding two brain regions.

[0069] After obtaining the time series characteristics and initial functional brain network of each brain region, the server can further capture the spatial information between brain regions through the GIN network. The classic graph convolution operation uses GNN to capture spatial information and construct a classification matrix. Since GNN does not work well when processing BOLD signals, GIN, which has been proven to have superior potential, is used instead. The expression of the GIN network is as follows: H 0 =AX, H 0 ∈R n×t ,

[0070] Where A∈R n*n is a brain network constructed using Pearson correlation, X is the time series characteristics of each brain region, is the feature information of the k-th layer input node of the graph isomorphic network, ∈ k is a learnable parameter, MLP k is the kth fully connected layer, Represents the information around the aggregation node, N i are the nodes adjacent to the i-th node in the initial functional brain network.

[0071] In order not to destroy the timing characteristics of the BOLD signal itself and make the GIN model pay more attention to the information of local time points, in one or more embodiments of the present invention, a one-dimensional convolution is used to replace the fully connected layer operation, and the GIN_C formula constructed in this way is as follows: conv2d() is the convolution function.

[0072] Therefore, the server can input the time series features of each brain region and the initial functional brain network into the graph isomorphic network, and perform spatial convolution on the time series features of each brain region according to the spatial relationship between the nodes in the initial functional brain network through the following formula:

[0073]

[0074] H 0 =AX,

[0075] Where Z is the time series feature of each brain region after spatial convolution, GIN_C( ) is the graph isomorphism network expression, and l is the preset maximum number of iterations of the graph isomorphism network.

[0076] After obtaining the time series characteristics of each brain region after spatial convolution, the server may cluster each brain region using differentiable graph pooling (diffpool). In one or more embodiments of the present invention, the server may cluster each brain region using the trained differentiable graph pooling according to the time series characteristics of each brain region after spatial convolution by the following formula, so as to eliminate brain regions irrelevant to potential brain diseases and aggregate brain regions with similar time series characteristics, and determine the cluster mapping matrix: S = softmax (GIN_C p (A,Z)),

[0077] Where S is the clustering mapping matrix, GIN_C p ( ) is a graph isomorphic network whose readout function is the pooling layer, and pool( ) is the pooling layer function.

[0078] Furthermore, the server can perform cluster mapping on the time series characteristics of each brain region after spatial convolution according to the cluster mapping matrix by the following formula, determine the time series characteristics of each brain region after brain region clustering, and construct the functional brain network after brain region clustering according to the time series characteristics of each brain region after clustering: Z'=S T Z, A'=S T A.S.

[0079] Where Z' is the time series characteristics of each brain region after brain region clustering, and A' is the functional brain network after brain region clustering.

[0080] For the process of differentiable graph pooling to obtain the mapping matrix, the mapping matrix is ​​made as sparse as possible through F regularization and information entropy loss, so that each row is in the form of unique hot encoding, making the aggregated brain areas independent, which is a hard aggregation method. The following loss function can be used for training constraints here:

[0081] Loss = αLoss en +(1-α)Loss LP ,

[0082]

[0083] In the formula, Loss LP To measure the cluster representation S T The difference between the brain network matrix reconstructed by S and the initial functional brain network matrix A, S T S represents the clustering similarity measurement matrix. Assuming that each node in the initial functional brain network corresponds to 116 brain regions, and each node in the functional brain network after brain region clustering corresponds to 64 brain regions, S is the clustering mapping matrix from 116 brain regions to 64 brain regions. TS can be multiplied to construct a 116×116 matrix. First of all, this matrix cannot deviate from the initial functional brain network. This is equivalent to adding the prior conditions of the initial functional brain network to make the functional brain network after brain region clustering as close to the initial functional brain network as possible. d is the normalization factor, and the value here can be the square of the number of brain regions after brain region clustering. Loss en The loss avoids overly certain attribution of data points by increasing the uncertainty in cluster assignment, that is, making the distribution more uniform rather than overly dependent on a certain cluster. ij It is the entropy measure for the non-negative part of the matrix S. ε is a very small positive number, which is usually used to avoid the problem of log(0) in the logarithmic function and ensure numerical stability.

[0084] based on Figure 1 The functional brain network construction method shown in the figure first obtains the user's resting-state brain functional magnetic resonance imaging, determines the time series characteristics of the brain regions according to the traditional brain region division, and constructs the initial functional brain network based on this, and then performs spatial convolution on the time series characteristics of each brain region according to the spatial relationship between the nodes in the initial functional brain network, so that the time series characteristics of each brain region after the spatial convolution are integrated with the information of its adjacent brain regions in the initial functional brain network, and then clusters the time series characteristics of each brain region after the spatial convolution to eliminate brain regions that are not related to potential brain diseases and aggregate brain regions with similar time series characteristics after the spatial convolution, so as to determine the cluster mapping matrix, and cluster map the time series characteristics of each brain region after the spatial convolution according to the cluster mapping matrix, and determine the time series characteristics of each brain region after the cluster mapping to construct the functional brain network after the brain regions are clustered.

[0085] The traditional brain region division has a high granularity, which leads to the complicated relationship between a large number of fine-grained brain regions, which is not conducive to direct observation and judgment. In general, not all brain regions are related to potential brain diseases. Therefore, on the one hand, the present invention aggregates traditional brain regions and eliminates brain regions that are not related to potential brain diseases during the aggregation mapping process, thereby improving the direct correlation between the functional brain network constructed thereby and potential brain diseases, and avoiding the interference of irrelevant brain regions in the characterization of potential brain diseases. On the other hand, the function of the brain region is characterized by its time series characteristics, and brain regions with similar functions are aggregated. By reducing the number of brain regions, the relationship between brain regions is clarified and centralized, thereby reducing the complexity of the functional brain network constructed thereby. The combination of these two aspects improves the degree of representation of the characteristics of potential brain diseases by the constructed functional brain network.

[0086] The present invention aggregates different fine-grained brain regions into coarse-grained brain regions by using spatial convolution. The characteristics of the relationship coefficients between coarse-grained brain regions are more significant, which is convenient for assisting doctors to identify the differences in functional brain networks between different subjects.

[0087] The present invention is based on the complementary information of multi-view data, with the purpose of constructing a good functional brain network, and then finding the biomarkers corresponding to brain diseases. The corresponding biomarkers can assist doctors in identifying brain diseases.

[0088] When applying the functional brain network construction method provided by the present invention, it is not necessary to Figure 1 The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and the present invention does not limit this.

[0089] In addition, in one or more embodiments of the present invention, the time series characteristics of each brain region after the brain regions are clustered can be further processed in the time dimension, that is, further coarsened in the time dimension to concentrate key information more, so as to obtain a brain network that can more directly characterize potential brain diseases.

[0090] For temporal learning, Transformer assisted by temporal convolution and spatial convolution can be used.

[0091] Temporal convolution is usually used to learn information about local time points and aggregate fine-grained time points into coarse-grained time points. For example, 60 time points in seconds are aggregated into one time point in minutes. The time point in minutes must contain information about 60 time points in seconds. How to integrate 60 seconds of information is the problem to be solved here. Whether it is averaging or maximum pooling, a lot of important information will inevitably be lost, and the characteristics of temporal information will be destroyed. Here, improved temporal convolution is used for aggregation, allowing the model to autonomously determine which information is useful and needs to be retained, and which information can be discarded.

[0092] The server can perform temporal convolution on the time series features of each brain region after brain region clustering using a temporal convolutional network through the following formula: Tconv(Z')=relu(bn(conv2d'(Z'))).

[0093] In the formula, Tconv( ) represents temporal convolution, conv2d'( ) is the convolution function and the convolution kernel size is (1, kernal_size), kernal_size is the size of the convolution kernel in the time dimension, which represents the number of aggregated time points, Z' is the time series feature of each brain region after brain region clustering, bn( ) is the batch normalization function, and relu( ) is the activation function. By limiting the convolution kernel size of the two-dimensional convolution to (1, kernal_size), it can be ensured that the convolution operation is only performed in the time dimension, thereby capturing the information in the time dimension and reconstructing the coarse-grained time information without losing the inherent logic of the time information.

[0094] The time series features of each brain region after time convolution are convolved in time and space by the following formula:

[0095] STGconv(Z')=Tconv(GIN_C(A',Z'))+bn(Tconv(Z')).

[0096] Where STGconv( ) represents the time-space convolution. The time-space convolution performs a gin operation before the time convolution, using spatial information as a priori to assist the convolution in capturing temporal information.

[0097] The Transformer model is used to integrate the characteristic information of a single brain region in the time dimension according to the time series characteristics of each brain region after time convolution and the time series characteristics of each brain region after time-space convolution through the following formula, and the coarse-grained time series characteristics of each brain region after time-space aggregation are obtained: Q = W q (Tconv(Z' T )), K = W k (STGconv(Z' T )), V = W v (Tconv(Z' T )),

[0098] Where Q is the query constructed by the Transformer model, and W q ( ) is the query constructor of the Transformer model, K is the key constructed by the Transformer model, and W k ( ) is the key constructor of the Transformer model, V is the value constructed by the Transformer model, and W v ( ) is the value constructor of the Transformer model, V out is the time series feature of each brain region after spatiotemporal aggregation, and n' is the number of brain regions after clustering.

[0099] Here we can construct the query Q, key K and value V through linear transformation. Z' is (brain area, time), and the transposed Z' T It is (time, brain area). The attention of a time point is constructed by dot product attention, and the global information of the data is optimized by the constructed time point × time point relationship matrix.

[0100] Then V out It can be input into the readout function. The readout function here can be average pooling, fully connected layer mapping, or orthogonal readout function. The readout function in this example is as follows:

[0101] In the formula, A r is the coarse-grained brain network after spatiotemporal aggregation, and Pearson ( ) is the Pearson correlation function.

[0102] That is, the Pearson correlation is used again to construct a coarse-grained brain network. In order to reduce data redundancy, only W is retained. r The upper half-angle matrix is ​​pulled into a vector as the output of the readout function.

[0103] In addition, the present invention also provides an embodiment of a method for constructing an executive function brain network, and the specific process is as follows: Figure 2 As shown:

[0104] (1) In this example, the Autism Brain Imaging Data Exchange (ABIDE) dataset from the New York site is used. There are 184 subjects in the dataset, including 105 normal people and 79 autistic patients. The functional brain images acquired by the magnetic resonance imaging device are converted and read. In order to ensure the stability of the time series, the first 10 time points of the magnetic resonance image are removed, and the image is corrected for head movement and time layer; the corrected image is aligned to the standard space, and the subject image is filtered using a 0.01-0.1HZ time bandpass filter to reduce the influence of heartbeat and breathing.

[0105] (2) Select a standardized brain partition template. In this example, the human brain is divided into 116 brain regions (90 brain regions of the cerebrum and 16 brain regions of the cerebellum) according to the AAL template. The 116 brain regions represent 116 nodes of the brain functional connection network. In fact, not all brain regions are functional. In this example, through model learning, brain regions that are not related to autism are eliminated, and brain regions with similar functions are aggregated, and finally 64 brain regions are combined.

[0106] (3) Calculate the time series average of all voxels in each brain region in the magnetic resonance image and extract the time series features corresponding to different standard brain regions. The extracted data can be expressed as X = [x1, x2, …, x i ,…,x n ]∈R 116×175 , where x i ∈R t represents the time series of the ith brain region, t represents the number of times in the time series, and n represents the number of brain regions. The initial brain network was constructed based on 116 brain regions by Pearson correlation.

[0107] (4) Based on the above ideas, a model is constructed, which consists of a hierarchical spatial learning module, a hierarchical temporal learning module, and a readout function.

[0108] (5) Hierarchical spatial learning module, as shown below:

[0109]

[0110] H 0 =AX,

[0111] S = softmax(GIN_C p (A,Z)),

[0112] Z'=S T Z, A'=S T A.S.

[0113] Among them, H 0 ∈R 116*175 represents the input bold signal, and GIN_C ( ) captures the spatial relationship between brain regions, which facilitates the subsequent aggregation of brain regions with similar functions. S is the clustering mapping matrix, the main purpose of which is to aggregate fine-grained brain regions into coarse-grained brain regions. p ( ) The last layer is the pooling layer, which takes R 116*175 Pooling is R 116*64 , normalized by softmax, we get S∈R 116*64 .

[0114] (6) Next, we construct a hierarchical time learning module:

[0115] Q=W q (relu(bn(conv2d'(Z' T ))))∈R 88*64 ,

[0116] K=W k (Tconv(GIN_C(Z' T ))+bn(Tconv(Z' T )))∈R 88*64 ,

[0117] V=W v (relu(bn(conv2d'(Z' T ))))∈R 88*64 ,

[0118]

[0119] For time information, two-dimensional convolution is mainly used for operation, and the convolution kernel can be controlled to (9,1) to ensure that the length of the time dimension output is 88.

[0120] (7) In step (6), coarse-grained brain region data is obtained. In this embodiment, Pearson correlation is used to reconstruct brain network A r ∈R 64*64 .

[0121] The above is a method for constructing a functional brain network provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding functional brain network construction device, such as Figure 3 shown.

[0122] Figure 3 A schematic diagram of a functional brain network construction device provided by the present invention, comprising:

[0123] An acquisition module 201 is used to determine the time series characteristics of each brain region based on the resting-state functional magnetic resonance imaging of patients with potential brain diseases and the brain region division of the automatic anatomical labeling atlas, and to construct an initial functional brain network;

[0124] The spatial convolution module 202 is used to perform spatial convolution on the time series features of each brain region according to the spatial relationship between the brain regions in the initial functional brain network, so that the time series features of each brain region after the spatial convolution are integrated with the information of its adjacent brain regions;

[0125] A clustering module 203 is used to cluster the time series characteristics of each brain region after spatial convolution, so as to eliminate brain regions that are not related to potential brain diseases and aggregate brain regions with similar time series characteristics after spatial convolution, and determine a cluster mapping matrix;

[0126] The construction module 204 is used to perform cluster mapping on the time series characteristics of each brain region after spatial convolution according to the cluster mapping matrix, and determine the time series characteristics of each brain region after cluster mapping to construct a functional brain network after brain region clustering.

[0127] For the specific definition of the functional brain network construction device, please refer to the definition of the functional brain network construction method above, which will not be repeated here. Each module in the above-mentioned functional brain network construction device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0128] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provided functional brain network construction method.

[0129] The present invention also provides Figure 4 The structural diagram of the computer device shown in FIG. Figure 4 As shown in the figure, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Provided functional brain network construction method.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0131] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the invention.

Claims

1. A method for constructing a functional brain network, characterized in that: include: Based on the resting-state functional magnetic resonance imaging of patients with potential brain diseases, the time series characteristics of each brain region are determined according to the brain region division of the automatic anatomical labeling atlas, and the initial functional brain network is constructed; According to the spatial relationship between brain regions in the initial functional brain network, the time series features of each brain region are spatially convolved, so that the time series features of each brain region after spatial convolution are integrated with the information of its adjacent brain regions; Clustering the time series features of each brain region after spatial convolution to eliminate brain regions that are not related to potential brain diseases and aggregate brain regions with similar time series features after spatial convolution to determine the cluster mapping matrix; The time series characteristics of each brain region after spatial convolution are clustered and mapped according to the cluster mapping matrix, and the time series characteristics of each brain region after cluster mapping are determined to construct a functional brain network after brain region clustering.

2. The method for constructing a functional brain network according to claim 1, characterized in that: The spatial convolution of the time series features of each brain region according to the spatial relationship between the brain regions in the initial functional brain network so that the time series features of each brain region after the spatial convolution are integrated with the information of its adjacent brain regions specifically includes: The time series features of each brain region and the initial functional brain network are input into the graph isomorphic network, and the time series features of each brain region are spatially convolved according to the spatial relationship between the nodes in the initial functional brain network using the following formula: Among them, Z is the time series characteristics of each brain region after spatial convolution, A is the initial functional brain network, X is the time series characteristics of each brain region, GIN_C() is the graph isomorphic network expression, is the characteristic information of the input node in the kth layer of the graph isomorphic network, N i is the node adjacent to the i-th node in the initial functional brain network, conv2d() is the convolution function, l is the preset maximum number of iterations of the graph isomorphic network, ∈ k are learnable parameters.

3. The method for constructing a functional brain network according to claim 2, characterized in that: The clustering of the time series features of each brain region after spatial convolution specifically includes: Differentiable graph pooling is used to cluster each brain region according to the time series characteristics of each brain region after spatial convolution using the following formula: S=softmax(GIN_C p (A,Z)), Among them, S is the clustering mapping matrix, GIN_C p () is a graph isomorphic network whose readout function is the pooling layer, and pool() is the pooling layer function.

4. The method for constructing a functional brain network according to claim 1, wherein: The method further comprises: The temporal convolution network is used to perform temporal convolution on the time series features of each brain region after brain region clustering through the following formula: Tconv(Z′)=relu(bn(conv2d′(Z′))); The time series features of each brain region after time convolution are convolved in time and space by the following formula: STGconv(Z′)=Tconv(GIN_C(A′,Z′))+bn(Tconv(Z′)); The Transformer model is used to integrate the characteristic information of a single brain region in the time dimension according to the time series characteristics of each brain region after time convolution and the time series characteristics of each brain region after time-space convolution through the following formula, and the coarse-grained time series characteristics of each brain region after time-space aggregation are obtained: Q=W q (Tconv(Z′ T )), W=W k (STGconv(Z′ T )), V=W v (Tconv(Z′ T )), Among them, Tconv() represents temporal convolution, conv2d′() is the convolution function and the convolution kernel size is (1, kernal_size), kernal_size is the size of the convolution kernel in the time dimension, Z′ is the time series feature of each brain region after brain region clustering, bn() is the batch normalization function, relu() is the activation function, STGconv() represents spatiotemporal convolution, A′ is the functional brain network after brain region clustering, Q is the query constructed by the Transformer model, and W q () is the query constructor of the Transformer model, K is the key constructed by the Transformer model, and W k () is the key constructor of the Transformer model, V is the value constructed by the Transformer model, W v () is the value constructor of the Transformer model, V out is the time series feature of each brain region after spatiotemporal aggregation, and n′ is the number of brain regions after clustering.

5. The method for constructing a functional brain network according to claim 1, wherein: The method of determining the time series characteristics of each brain region based on the resting-state functional magnetic resonance imaging of patients with potential brain diseases and the brain region division of the automatic anatomical labeling atlas specifically includes: Performing slice time correction and head motion correction on the user's resting-state brain functional magnetic resonance image, and removing the influence of the ventricle, white matter signals and the high-order effect of head motion from the corrected resting-state brain functional magnetic resonance image, to obtain a resting-state brain functional magnetic resonance image after preliminary preprocessing; The resting-state functional magnetic resonance images after preliminary preprocessing are registered to the standard space, and the registered resting-state functional magnetic resonance images are subjected to 0.01-0.1 Hz time bandpass filtering to reduce the influence of heartbeat and breathing, so as to obtain the final resting-state functional magnetic resonance images after preprocessing; According to the automatic anatomical labeling atlas, the brain is divided into multiple brain regions, and the average time series characteristics of each brain region are extracted based on the final preprocessed resting-state functional magnetic resonance images as the time series characteristics of each brain region.

6. A functional brain network construction device, characterized in that: include: An acquisition module is used to determine the time series characteristics of each brain region based on the resting-state functional magnetic resonance imaging of patients with potential brain diseases according to the brain region division of the automatic anatomical labeling atlas, and to construct an initial functional brain network; The spatial convolution module is used to perform spatial convolution on the time series features of each brain region according to the spatial relationship between the nodes in the initial functional brain network, so that the time series features of each brain region after spatial convolution are integrated with the information of its adjacent nodes; A clustering module is used to cluster the time series features of each brain region after spatial convolution, so as to eliminate brain regions that are not related to potential brain diseases and aggregate brain regions with similar time series features after spatial convolution to determine the cluster mapping matrix; The construction module is used to cluster and map the time series characteristics of each brain region after spatial convolution according to the cluster mapping matrix, determine the time series characteristics of each brain region after cluster mapping to construct a functional brain network after brain region clustering.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

Images