A method for image classification based on functional magnetic resonance imaging

Abstract

The present invention provides a method for image classification based on functional magnetic resonance imaging, which includes obtaining functional magnetic resonance imaging of the human brain, extracting time series of each target region of interest based on a predetermined target region of interest, and further generating a functional connectivity matrix; importing the functional connectivity matrix into a trained deep learning classification model to obtain a classification category; wherein the deep learning classification model is composed of a sub-connection mask learning module, a Transformer encoder, and a classifier; the sub-connection learning module is used to dynamically generate a Mask matrix for removing invalid redundant connections from the functional connectivity matrix based on the multi-head self-attention mechanism; the Transformer encoder is used to perform a dot product operation on the Mask matrix and the functional connectivity matrix to obtain a sub-connection network and perform feature encoding; the classifier is used to perform category recognition on the encoded sub-connection network. Implementing the present invention can reduce invalid redundant connections in the brain functional network, thereby achieving accurate classification of specific targets.

Description

Technical Field

[0001] The present invention relates to the field of computer image technology, and in particular to a method for image classification based on functional magnetic resonance imaging. Background Art

[0002] Compared with other imaging technologies, functional magnetic resonance imaging has higher resolution and more accurate positioning ability, so it is increasingly widely used in medical neuroimaging. Currently, in the process of image classification based on functional magnetic resonance imaging, brain functional connectivity analysis has become increasingly popular, providing strong medical data support for the recognition of the final image categories to a certain extent.

[0003] However, existing brain functional connectivity analysis methods based on functional magnetic resonance imaging focus on using global connections to capture relevant information, and there are many ineffective redundant connections, so accurate classification of specific targets cannot be achieved ultimately.

[0004] Therefore, it is necessary to improve the existing brain functional connectivity analysis in image classification based on functional magnetic resonance imaging, which can reduce the ineffective redundant connections in the brain functional network, so as to achieve accurate classification of specific targets. Summary of the Invention

[0005] The technical problem to be solved by the embodiments of the present invention is to provide a method for image classification based on functional magnetic resonance imaging, which can reduce the ineffective redundant connections in the brain functional network, so as to achieve accurate classification of specific targets.

[0006] To solve the above technical problem, the embodiments of the present invention provide a method for image classification based on functional magnetic resonance imaging, and the method includes the following steps:

[0007] Obtain the functional magnetic resonance imaging of the human brain, and based on a predetermined target region of interest, extract the time series of each target region of interest from the functional magnetic resonance imaging, and further calculate the Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connectivity matrix;

[0008] Import the functional connection matrix into a trained deep learning classification model to obtain the corresponding category as normal or abnormal; wherein, the deep learning classification model is composed of a sub-connection mask learning module, a Transformer encoder, and a classifier; the sub-connection learning module is used to dynamically generate a Mask matrix that removes invalid redundant connections based on the multi-head self-attention mechanism for the functional connection matrix; the Transformer encoder is used to perform a dot product operation on the Mask matrix obtained by the sub-connection learning module and the functional connection matrix to obtain a sub-connection network, and further perform feature encoding on the sub-connection network; the classifier is used to perform category recognition on the sub-connection network after feature encoding.

[0009] Among them, the specific steps for the sub-connection learning module to dynamically generate a Mask matrix that removes invalid redundant connections based on the multi-head self-attention mechanism for the functional connection matrix include:

[0010] Perform self-attention encoding on the functional connection matrix to obtain an attention matrix, and splice the attention matrix through a multi-head fully connected layer to obtain multiple Mask matrices;

[0011] Perform weighted fusion on the obtained multiple Mask matrices to obtain a fused Mask matrix, and further perform binarization processing on the obtained fused Mask matrix to obtain a Mask matrix that removes invalid redundant connections.

[0012] Among them, through the formula , perform binarization processing on the obtained fused Mask matrix to obtain a Mask matrix that removes invalid redundant connections ; where

[0013] is the fused Mask matrix, ij represents the corresponding row and column elements in

[0014] Among them, the specific steps for obtaining the functional magnetic resonance imaging of the human brain, and based on a predetermined target region of interest, extracting the time series of each target region of interest from the functional magnetic resonance imaging, and further calculating the Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connection matrix include:

[0015] Using brain imaging processing software, perform standardization processing on the functional magnetic resonance imaging, and map the standardized functional magnetic resonance imaging into the Montreal Neurological Institute (MNI) standard space through non-linear registration to obtain the functional magnetic resonance imaging defined by pixels;

[0016] Select a standardized atlas to define the target region of interest, and generate a binary mask through thresholding;

[0017] According to the generated binary mask, in the functional magnetic resonance imaging defined by pixels, extract the time series signals of each target region of interest in the functional magnetic resonance imaging, and remove noise, de-trend and standardize through band-pass filtering to obtain the time series of each target region of interest;

[0018] Use the Pearson correlation coefficient to calculate the functional connection strength between the time series of any two target regions of interest; among them, the calculation method of the Pearson correlation coefficient is specifically as follows: first calculate the mean of the time series, then calculate their covariance, then calculate the standard deviation, and finally standardize the covariance to obtain the functional connection coefficient between each pair;

[0019] Organize all the calculated functional connection strengths to obtain a functional connection matrix.

[0020] Implementing the embodiments of the present invention has the following beneficial effects:

[0021] The present invention forms a functional connection matrix from the functional magnetic resonance imaging of the human brain, and uses a sub-connection mask learning module with multi-head self-attention in the deep learning classification model to learn the sub-functional connection network with strong correlation in the whole-brain functional connection network to remove invalid redundant connections, and then uses a Transformer encoder to encode it and send it to a classifier for classification, so as to reduce the invalid redundant connections of the brain functional network, thereby achieving the purpose of accurate classification of specific targets. Description of the Drawings

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, obtaining other drawings based on these drawings still belongs to the scope of the present invention.

[0023] Figure 1 It is a flowchart of a method for analyzing brain functional network connections in functional magnetic resonance imaging provided by an embodiment of the present invention;

[0024] Figure 2 This is an application scenario diagram of a method for analyzing brain functional network connections in functional magnetic resonance imaging provided by an embodiment of the present invention. Detailed implementation manners

[0025] To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0026] As Figure 1 shown, in an embodiment of the present invention, a method for analyzing brain functional network connections in functional magnetic resonance imaging is provided. The method includes the following steps:

[0027] Step S1: Obtain a functional magnetic resonance imaging of a human brain, and based on a predetermined target region of interest, extract the time series of each target region of interest from the functional magnetic resonance imaging, and further calculate the Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connectivity matrix;

[0028] Specifically, first, obtain a functional magnetic resonance imaging of a human brain, and use brain imaging processing software to perform standardization processing on the functional magnetic resonance imaging, and further map the standardized functional magnetic resonance imaging into the Montreal Neurological Institute (MNI) standard space (resolution 1×1×1 mm³) through non-linear registration to obtain a functional magnetic resonance imaging defined by pixels;

[0029] Select a standardized atlas to define the target region of interest, and generate a binary mask through thresholding; for example, in the functional magnetic resonance imaging of a human brain, use voxel-based morphometry (VBM) to segment gray matter, white matter, and cerebrospinal fluid components as the target region of interest;

[0030] According to the generated binary mask, extract the time series signals of each target region of interest in the functional magnetic resonance imaging defined by pixels, and remove noise, de-trend, and standardize through band-pass filtering (0.01 - 0.1 Hz) to obtain the time series of each target region of interest;

[0031] Use the Pearson correlation coefficient to calculate the functional connectivity strength between the time series of any two target regions of interest; wherein, the calculation method of the Pearson correlation coefficient is specifically as follows: first calculate the time series mean, then calculate their covariance, then calculate the standard deviation, and finally standardize the covariance to obtain the functional connectivity coefficient between each pair;

[0032] Organize all the calculated functional connectivity strengths to obtain a functional connectivity matrix

[0033] Step S2: Import the functional connection matrix into the trained deep learning classification model to obtain the corresponding category as normal or abnormal; wherein, the deep learning classification model is composed of a sub-connection mask learning module, a Transformer encoder, and a classifier; the sub-connection learning module is used to dynamically generate a Mask matrix removing invalid redundant connections based on the multi-head self-attention mechanism from the functional connection matrix; the Transformer encoder is used to perform a dot product operation on the Mask matrix obtained by the sub-connection learning module and the functional connection matrix to obtain a sub-connection network, and further perform feature encoding on the sub-connection network; the classifier is used to perform category recognition on the sub-connection network after feature encoding.

[0034] Specifically, first, a deep learning classification model composed of a sub-connection mask learning module, a Transformer encoder, and a classifier is constructed.

[0035] At this time, the input of the sub-connection learning module is the functional connection matrix X, and the output is the Mask matrix removing invalid redundant connections. Among them, the specific steps for this module to dynamically generate a Mask matrix removing invalid redundant connections from the functional connection matrix based on the multi-head self-attention mechanism are as follows:

[0036] Perform self-attention encoding on the functional connection matrix to obtain an attention matrix, and through a multi-head fully connected layer, splice the attention matrix to obtain multiple Mask matrices;

[0037] Perform weighted fusion on the obtained multiple Mask matrices to obtain a fused Mask matrix, and further perform binarization processing on the obtained fused Mask matrix to obtain a Mask matrix removing invalid redundant connections.

[0038] For example, use multi-head self-attention to obtain an attention map A :

[0039]

[0040]

[0041] Among them, Q h is the query matrix ( Query ), K h is the key matrix ( Key ), V h is the value matrix ( Value ), d is the dimension of the query vector, softmaxis the probability distribution function normalized along the rows, A h is the output feature of the h -th attention head. Then, the attention matrices of multiple heads are concatenated to obtain the final attention map A . In this example h = 4;

[0042] Design a multi-mask fusion strategy, that is, after concatenating the calculation results of the multi-head self-attention, use multiple linear mappings and sigmoid functions to obtain multiple groups of masks. Then we fuse all the masks using learnable weight parameters:

[0043]

[0044] where, α c are learnable weight parameters, represents the fused Mask matrix;

[0045] To further remove invalid connections, through the formula , perform binarization on the fused Mask matrix to obtain a Mask matrix that removes invalid redundant connections ;

[0046] where, ij represents the corresponding row and column elements in

[0047] At this time, the Transformer encoder performs a dot product operation on the Mask matrix obtained by the sub-connection learning module and the functional connection matrix to obtain a sub-connection network, and further encodes the features of the sub-connection network; that is, the input of the Transformer encoder is Xa after the dot product of the functional connection matrix X and the Mask matrix, and the output is Xb after feature encoding.

[0048] At this time, the classifier performs class recognition on the sub-connection network after feature encoding by the Transformer encoder, that is, the input of the classifier is Xb after feature encoding, and the output classes are normal Y1 or abnormal Y2.

[0049] Next, train the deep learning classification model, and the specific process is as follows:

[0050] Step 1: Obtain functional magnetic resonance images of multiple human brains from the dataset and process them to obtain multiple groups of training functional connection matrices X;

[0051] Step 2: Input the functional connection matrix X into the sub-connection mask learning module to obtain the Mask matrix;

[0052] Step 3: Multiply the Mask matrix by X and continue to input it into the sub-connection mask learning module. Repeat Step 2 multiple times to obtain the final output Mask matrix;

[0053] Step 4: Multiply the obtained Mask matrix by the original training functional connection matrix X to obtain Xa, and input it into the Transformer encoder for feature encoding, and output to obtain Xb;

[0054] Step 5: Input the obtained Xb into the classifier to obtain the prediction probability P , and calculate the loss function:

[0055]

[0056] where L CE is the binary cross-entropy loss function, Y is the one-hot encoding of the true label, P is the predicted probability distribution output by the classifier;

[0057] Step 6: Repeat Steps 1 to 5 until e iterations are completed, so as to obtain a trained deep learning classification model; where e is the number of training epochs given in advance; in this example, e = 100.

[0058] Finally, import the functional connection matrix obtained in Step S1 into the above-trained deep learning classification model, and it can quickly identify the corresponding category as normal or abnormal.

[0059] As Figure 2 shown, the application scenario of a method for analyzing brain functional network connections in functional magnetic resonance imaging provided in the embodiment of the present invention is further described as follows:

[0060] Read the functional magnetic resonance imaging of a single human brain, process it into multiple time series divided based on the target region of interest, and calculate to obtain a 200*200 functional connection matrix;

[0061] Perform self-attention encoding on the 200*200 functional connection matrix to obtain an attention matrix, then use this attention matrix to obtain multiple Mask matrices through a multi-head fully connected layer, and after weighted fusion, obtain a final 200*200-sized Mask matrix as the output of the sub-connection learning module;

[0062] Then, use this Mask matrix to perform a dot product operation with the 200*200 original functional connection matrix to obtain a 200*200 sub-connection network, where the irrelevant connections have been masked;

[0063] Then, the sub-connection network is fed into a Transformer encoder for feature encoding to obtain the sub-connection network after feature encoding;

[0064] Finally, the sub-connection network after feature encoding is fed into a classifier composed of multiple linear and activation layers to obtain the corresponding classification category.

[0065] Implementing the embodiments of the present invention has the following beneficial effects:

[0066] The present invention forms a functional connection matrix from the functional magnetic resonance imaging of the human brain and uses a sub-connection mask learning module with multi-head self-attention in a deep learning classification model to learn the sub-functional connection networks with strong correlations in the whole-brain functional connection network to remove invalid redundant connections, and then uses a Transformer encoder to encode it and feeds it into a classifier for classification, achieving the reduction of invalid redundant connections in the brain functional network, thereby realizing the accurate classification of specific targets.

[0067] Those of ordinary skill in the art can understand that all or part of the steps in implementing the above method embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium, such as ROM / RAM, disk, optical disc, etc.

[0068] The above-disclosed are only the preferred embodiments of the present invention, and of course, the scope of rights of the present invention cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope covered by the present invention.

Claims

1. A method for image classification based on functional magnetic resonance imaging, characterized in that, The method includes the following steps: Obtain the functional magnetic resonance imaging of the human brain, and based on a predetermined target region of interest, extract the time series of each target region of interest from the functional magnetic resonance imaging, and further calculate the Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connectivity matrix; Import the functional connectivity matrix into a trained deep learning classification model to obtain a corresponding category of normal or abnormal; wherein, the deep learning classification model is composed of a sub-connectivity learning module, a Transformer encoder, and a classifier; the sub-connectivity learning module is used to dynamically generate a Mask matrix that removes invalid redundant connections based on the multi-head self-attention mechanism; the Transformer encoder is used to perform a dot product operation on the Mask matrix obtained by the sub-connectivity learning module and the functional connectivity matrix to obtain a sub-connectivity network, and further perform feature encoding on the sub-connectivity network; the classifier is used to perform category recognition on the feature-encoded sub-connectivity network; The specific steps for the sub-connectivity learning module to dynamically generate a Mask matrix that removes invalid redundant connections based on the multi-head self-attention mechanism include: Perform self-attention encoding on the functional connectivity matrix to obtain an attention matrix, and splice the attention matrix through a multi-head fully connected layer to obtain multiple Mask matrices; Perform weighted fusion on the obtained multiple Mask matrices to obtain a fused Mask matrix, and further perform binarization processing on the obtained fused Mask matrix to obtain a Mask matrix that removes invalid redundant connections.

2. The method for image classification based on functional magnetic resonance imaging according to claim 1, characterized in that, Through the formula , perform binarization processing on the obtained fused Mask matrix to obtain a Mask matrix with invalid redundant connections removed ; among them, is the fused Mask matrix, ij denotes the corresponding row and column elements in 3. The method for image classification based on functional magnetic resonance imaging according to claim 1, wherein The specific steps for obtaining the functional magnetic resonance imaging of the human brain, and based on a predetermined target region of interest, extracting the time series of each target region of interest from the functional magnetic resonance imaging, and further calculating the Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connectivity matrix include: Use brain imaging processing software to perform normalization processing on the functional magnetic resonance imaging, and map the normalized functional magnetic resonance imaging to the Montreal Neurological Institute (MNI) standard space through non-linear registration to obtain the functional magnetic resonance imaging defined by pixels; Select a standardized atlas to define the target region of interest and generate a binary mask through thresholding; According to the generated binary mask, extract the time series signals of each target region of interest in the functional magnetic resonance imaging defined by pixels, and remove noise, remove linear trends, and perform normalization processing through band-pass filtering to obtain the time series of each target region of interest; The Pearson correlation coefficient is used to calculate the functional connectivity strength between the time series of any two target regions of interest. Specifically, the calculation method of the Pearson correlation coefficient is as follows: first, calculate the mean of the time series, then calculate their covariance, then calculate the standard deviation, and finally standardize the covariance to obtain the functional connectivity coefficient between each pair. Organize all the calculated functional connectivity strengths to obtain a functional connectivity matrix.

Images