Image classification method based on functional nuclear magnetic resonance imaging

Abstract

The invention provides a functional nuclear magnetic resonance imaging-based image classification method, which comprises the following steps of: acquiring functional nuclear magnetic resonance imaging of a human brain, extracting a time sequence of each target region of interest based on preset target regions of interest, and further generating a functional connection matrix; importing the function connection matrix into a trained deep learning classification model to obtain a classification category; wherein the deep learning classification model is composed of a sub-connection mask learning module, a Transform encoder and a classifier; the sub-connection learning module is used for generating the function connection matrix into a Mask matrix with invalid redundant connection removed based on the multi-head self-attention mechanism dynamic state; the Transform encoder is used for performing dot product operation on the Mask matrix and the functional connection matrix to obtain a sub-connection network and performing feature coding; and the classifier is used for carrying out category identification on the coded sub-connection network. By implementing the method and the device, invalid redundant connection of the brain function network can be reduced, so that accurate classification of specific targets is realized.

Description

Technical Field

[0001] The present invention relates to the field of computer image technology, and in particular to a method for image classification based on functional magnetic resonance imaging. Background Art

[0002] Compared with other imaging technologies, functional magnetic resonance imaging has higher resolution and precise positioning capabilities, so it is increasingly used in medical neuroimaging. At present, in the process of image classification based on functional magnetic resonance imaging, brain functional connectivity analysis is becoming more and more popular, which to a certain extent provides strong medical data support for the identification of the final image category.

[0003] However, existing functional MRI-based brain functional connectivity analysis methods focus on using global connections to capture relevant information. There are many invalid redundant connections, and therefore it is ultimately impossible to accurately classify specific targets.

[0004] Therefore, it is necessary to improve the existing brain functional connectivity analysis in image classification based on functional magnetic resonance imaging, so as to reduce the invalid redundant connections of brain functional networks and achieve accurate classification of specific targets. Summary of the invention

[0005] The technical problem to be solved by the embodiments of the present invention is to provide a method for image classification based on functional magnetic resonance imaging, which can reduce the invalid redundant connections of the brain functional network, thereby achieving accurate classification of specific targets.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a method for image classification based on functional magnetic resonance imaging, the method comprising the following steps: Acquire a functional magnetic resonance imaging of the human brain, and based on a predetermined target region of interest, extract a time series of each target region of interest from the functional magnetic resonance imaging, and further calculate a Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connectivity matrix; The functional connection matrix is ​​imported into a trained deep learning classification model to obtain a corresponding category of normal or abnormal; wherein the deep learning classification model is composed of a sub-connection mask learning module, a Transformer encoder and a classifier; the sub-connection learning module is used to dynamically generate a Mask matrix from the functional connection matrix that removes invalid redundant connections based on a multi-head self-attention mechanism; the Transformer encoder is used to perform a dot product operation on the Mask matrix obtained by the sub-connection learning module and the functional connection matrix to obtain a sub-connection network, and further feature encode the sub-connection network; the classifier is used to perform category recognition on the feature-encoded sub-connection network.

[0007] The sub-connection learning module generates a Mask matrix that removes invalid redundant connections from the functional connection matrix based on the dynamic multi-head self-attention mechanism, and the specific steps include: Performing self-attention encoding on the functional connection matrix to obtain an attention matrix, and concatenating the attention matrix through a multi-head fully connected layer to obtain multiple Mask matrices; The obtained multiple Mask matrices are weighted fused to obtain a fused Mask matrix, and the obtained fused Mask matrix is ​​further binarized to obtain a Mask matrix with invalid redundant connections removed.

[0008] Among them, through the formula , binarize the fused Mask matrix to obtain a Mask matrix that removes invalid redundant connections ;in, is the fused Mask matrix, ij express The corresponding row and column elements in .

[0009] The specific steps of acquiring a functional magnetic resonance imaging of the human brain, extracting a time series of each target region of interest from the functional magnetic resonance imaging based on a predetermined target region of interest, and further calculating the Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connectivity matrix include: Using brain image processing software to standardize the functional magnetic resonance imaging, and mapping the standardized functional magnetic resonance imaging into the Montreal Neurological Institute (MNI) standard space by nonlinear registration to obtain a pixel-defined functional magnetic resonance imaging; Select a standardized atlas to define the target region of interest and generate a binary mask through thresholding; According to the generated binary mask, in the pixel-defined functional magnetic resonance imaging, a time series signal of each target region of interest in the functional magnetic resonance imaging is extracted, and the time series of each target region of interest is obtained by bandpass filtering to remove noise, delinear trend and normalization. The Pearson correlation coefficient is used to calculate the functional connection strength between the time series of any two target regions of interest; wherein the Pearson correlation coefficient is calculated by first calculating the mean of the time series, then calculating their covariance, then calculating the standard deviation, and finally standardizing the covariance to obtain the functional connection coefficient between the two; All calculated functional connection strengths are organized to obtain a functional connection matrix.

[0010] Implementing the embodiments of the present invention has the following beneficial effects: The present invention forms a functional connection matrix from the functional magnetic resonance imaging of the human brain, and uses a sub-connection mask learning module of multi-head self-attention in a deep learning classification model to learn the sub-functional connection network with strong correlation in the whole-brain functional connection network to remove invalid redundant connections, and then uses a Transformer encoder to encode it and send it to a classifier for classification, so as to reduce the invalid redundant connections of the brain functional network, thereby achieving the purpose of accurately classifying specific targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, without paying creative labor, other drawings obtained based on these drawings still belong to the scope of the present invention.

[0012] Figure 1 A flowchart of a method for analyzing brain functional network connectivity in functional magnetic resonance imaging provided by an embodiment of the present invention; Figure 2 A diagram of an application scenario of a method for analyzing brain functional network connectivity in functional magnetic resonance imaging provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0014] like Figure 1 As shown in the figure, a method for analyzing brain functional network connectivity in functional magnetic resonance imaging is provided in an embodiment of the present invention, and the method comprises the following steps: Step S1, obtaining a functional magnetic resonance imaging of the human brain, and based on a predetermined target region of interest, extracting a time series of each target region of interest from the functional magnetic resonance imaging, and further calculating a Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connectivity matrix; The specific process is as follows: first, obtain the functional magnetic resonance imaging of the human brain, and use brain image processing software to standardize the functional magnetic resonance imaging, and further map the standardized functional magnetic resonance imaging to the Montreal Neurological Institute (MNI) standard space (resolution 1×1×1 mm³) through nonlinear registration to obtain the functional magnetic resonance imaging defined by pixels; A standardized atlas is selected to define the target region of interest, and a binary mask is generated by thresholding; for example, in functional magnetic resonance imaging of the human brain, voxel-based morphometry (VBM) is used to segment gray matter, white matter, and cerebrospinal fluid components as the target region of interest; According to the generated binary mask, the time series signal of each target region of interest in the functional MRI was extracted in the pixel-defined functional MRI, and the time series of each target region of interest was obtained by bandpass filtering (0.01-0.1 Hz) to remove noise, delinear trend and normalization; The Pearson correlation coefficient is used to calculate the functional connection strength between the time series of any two target regions of interest; wherein the Pearson correlation coefficient is calculated by first calculating the mean of the time series, then calculating their covariance, then calculating the standard deviation, and finally standardizing the covariance to obtain the functional connection coefficient between the two; Organize all calculated functional connection strengths to obtain the functional connection matrix Step S2, importing the functional connection matrix into a trained deep learning classification model to obtain the corresponding category of normal or abnormal; wherein the deep learning classification model is composed of a sub-connection mask learning module, a Transformer encoder and a classifier; the sub-connection learning module is used to dynamically generate a Mask matrix from the functional connection matrix to remove invalid redundant connections based on a multi-head self-attention mechanism; the Transformer encoder is used to perform a dot product operation on the Mask matrix obtained by the sub-connection learning module and the functional connection matrix to obtain a sub-connection network, and further feature encode the sub-connection network; the classifier is used to perform category recognition on the feature-encoded sub-connection network.

[0015] The specific process is as follows: first, a deep learning classification model consisting of a sub-connection mask learning module, a Transformer encoder and a classifier is constructed.

[0016] At this time, the input of the sub-connection learning module is the functional connection matrix X, and the output is the Mask matrix that removes invalid redundant connections. Among them, the specific steps of this module to generate the Mask matrix that removes invalid redundant connections from the functional connection matrix based on the dynamic multi-head self-attention mechanism are as follows: The functional connection matrix is ​​self-attention encoded to obtain the attention matrix, and the attention matrix is ​​spliced ​​through a multi-head fully connected layer to obtain multiple mask matrices; The obtained multiple Mask matrices are weighted fused to obtain a fused Mask matrix, and the obtained fused Mask matrix is ​​further binarized to obtain a Mask matrix with invalid redundant connections removed.

[0017] For example, using multi-head self-attention to obtain the attention map A : in, Q h is the query matrix ( Query ), K h is the key matrix ( Key ), V h is the value matrix ( Value ), d is the dimension of the query vector, softmax is the probability distribution function normalized along the row, A h For the h The output features of the attention heads are then concatenated to obtain the final attention map. A In this example h =4; We design a multi-mask fusion strategy, that is, after splicing the calculation results of multiple self-attention heads, we use multiple linear mappings and sigmoid functions to obtain multiple sets of masks. Then we use learnable weight parameters to fuse all masks: in, α c is the learnable weight parameter, Represents the fused Mask matrix; In order to further remove invalid connections, the formula , binarize the fused Mask matrix to obtain a Mask matrix that removes invalid redundant connections ; in, ij express The corresponding row and column elements in .

[0018] At this time, the Transformer encoder performs a dot product operation on the Mask matrix obtained by the sub-connection learning module and the functional connection matrix to obtain a sub-connection network, and further performs feature encoding on the sub-connection network; that is, the input of the Transformer encoder is Xa after the dot product of the functional connection matrix X and the Mask matrix, and the output is Xb after feature encoding.

[0019] At this time, the classifier performs category recognition on the sub-connected network after feature encoding of the Transformer encoder, that is, the input of the classifier is Xb after feature encoding, and the output category is normal Y1 or abnormal Y2.

[0020] Next, the deep learning classification model is trained. The specific process is as follows: Step 1: obtain multiple functional magnetic resonance imaging images of human brain from the data set, and process them to obtain multiple sets of training functional connection matrices X; Step 2: Input the functional connection matrix X into the sub-connection mask learning module to obtain the Mask matrix; Step 3: Multiply the Mask matrix by X and continue to input the sub-connection mask learning module. Repeat step 2 multiple times to obtain the final output Mask matrix; Step 4: Multiply the obtained Mask matrix with the original training feature connection matrix X to obtain Xa, and input it into the Transformer encoder for feature encoding, and output Xb; Step 5: Input the obtained Xb into the classifier to obtain the predicted probability P , and calculate the loss function: in L CE is the binary cross entropy loss function, Y is the one-hot encoding of the true label, P is the predicted probability distribution of the classifier output; Step 6: Repeat steps 1 to 5 until e iterations are completed, thereby obtaining a trained deep learning classification model; where e is the number of training rounds given in advance; in this example, e=100.

[0021] Finally, the functional connectivity matrix obtained in step S1 is imported into the above-mentioned trained deep learning classification model, which can quickly identify the corresponding category as normal or abnormal.

[0022] like Figure 2 As shown, the application scenario of the method for analyzing brain functional network connectivity in functional magnetic resonance imaging provided in an embodiment of the present invention is further described as follows: Read the functional magnetic resonance imaging of a single human brain, process it into multiple time series based on the target of interest, and calculate the functional connection matrix 200*200; The 200*200 functional connection matrix is ​​self-attention encoded to obtain the attention matrix, and then the attention matrix is ​​used to obtain multiple mask matrices through a multi-head fully connected layer. After weighted fusion, the final 200*200 size mask matrix is ​​obtained as the output of the sub-connection learning module; Next, the Mask matrix is ​​used to perform a dot product operation with the 200*200 original functional connection matrix to obtain a 200*200 sub-connection network in which irrelevant connections have been masked; Then, the sub-connection network is sent to the Transformer encoder for feature encoding to obtain the feature-encoded sub-connection network; Finally, the feature-encoded sub-connected network is fed into a classifier consisting of multiple linear and activation layers to obtain the corresponding classification category.

[0023] Implementing the embodiments of the present invention has the following beneficial effects: The present invention forms a functional connection matrix from the functional magnetic resonance imaging of the human brain, and uses a sub-connection mask learning module of multi-head self-attention in a deep learning classification model to learn the sub-functional connection network with strong correlation in the whole-brain functional connection network to remove invalid redundant connections, and then uses a Transformer encoder to encode it and send it to a classifier for classification, so as to reduce the invalid redundant connections of the brain functional network, thereby achieving the purpose of accurately classifying specific targets.

[0024] A person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as ROM / RAM, disk, CD-ROM, etc.

[0025] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for image classification based on functional magnetic resonance imaging, characterized in that: The method comprises the following steps: Acquire a functional magnetic resonance imaging of the human brain, and based on a predetermined target region of interest, extract a time series of each target region of interest from the functional magnetic resonance imaging, and further calculate a Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connectivity matrix; The functional connection matrix is ​​imported into a trained deep learning classification model to obtain a corresponding category of normal or abnormal; wherein the deep learning classification model is composed of a sub-connection mask learning module, a Transformer encoder and a classifier; the sub-connection learning module is used to dynamically generate a Mask matrix from the functional connection matrix that removes invalid redundant connections based on a multi-head self-attention mechanism; the Transformer encoder is used to perform a dot product operation on the Mask matrix obtained by the sub-connection learning module and the functional connection matrix to obtain a sub-connection network, and further feature encode the sub-connection network; the classifier is used to perform category recognition on the feature-encoded sub-connection network.

2. The method for image classification based on functional magnetic resonance imaging according to claim 1, characterized in that: The sub-connection learning module generates a Mask matrix that removes invalid redundant connections from the functional connection matrix based on the dynamic multi-head self-attention mechanism, and the specific steps include: Performing self-attention encoding on the functional connection matrix to obtain an attention matrix, and concatenating the attention matrix through a multi-head fully connected layer to obtain multiple Mask matrices; The obtained multiple Mask matrices are weighted fused to obtain a fused Mask matrix, and the obtained fused Mask matrix is ​​further binarized to obtain a Mask matrix with invalid redundant connections removed.

3. The method for image classification based on functional magnetic resonance imaging as claimed in claim 2, characterized in that: By formula , binarize the fused Mask matrix to obtain a Mask matrix that removes invalid redundant connections ;in, is the fused Mask matrix, ij express The corresponding row and column elements in .

4. The method for image classification based on functional magnetic resonance imaging according to claim 1, characterized in that: The specific steps of acquiring a functional magnetic resonance imaging of the human brain, extracting a time series of each target region of interest from the functional magnetic resonance imaging based on a predetermined target region of interest, and further calculating a Pearson correlation coefficient between the time series of any two target regions of interest to generate a functional connectivity matrix include: Using brain image processing software to standardize the functional magnetic resonance imaging, and mapping the standardized functional magnetic resonance imaging into the Montreal Neurological Institute (MNI) standard space by nonlinear registration to obtain a pixel-defined functional magnetic resonance imaging; Select a standardized atlas to define the target region of interest and generate a binary mask through thresholding; According to the generated binary mask, in the pixel-defined functional magnetic resonance imaging, a time series signal of each target region of interest in the functional magnetic resonance imaging is extracted, and the time series of each target region of interest is obtained by bandpass filtering to remove noise, delinear trend and normalization. The Pearson correlation coefficient is used to calculate the functional connection strength between the time series of any two target regions of interest; wherein the Pearson correlation coefficient is calculated by first calculating the mean of the time series, then calculating their covariance, then calculating the standard deviation, and finally standardizing the covariance to obtain the functional connection coefficient between the two; All calculated functional connection strengths are organized to obtain a functional connection matrix.

Images