End-to-end brain image data processing method and device based on deep learning

Abstract

The present invention discloses an end-to-end brain image data processing method based on deep learning, including: obtaining control group brain image data and test group brain image data; performing feature extraction on the control group brain image data and the test group brain image data respectively, and obtaining a feature matrix to be trained and a feature matrix to be detected; constructing an end-to-end convolutional neural network classification model; training the end-to-end convolutional neural network classification model using the feature matrix to be trained; processing the feature matrix to be detected using the trained end-to-end convolutional neural network classification model to obtain a detection result. The method provided by the present invention integrates the entire analysis process into a whole, and realizes the classification result based on the deep learning algorithm, so that the analysis algorithm is more coherent, and the output of the model directly reflects the information expressed by the input data, thereby making the detection result more accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical equipment, and specifically relates to an end-to-end brain image data processing method and device based on deep learning. Background Art

[0002] Crohn's disease (CD) is a chronic nonspecific granulomatous inflammatory bowel disease, with the main clinical manifestations of recurrent abdominal pain, diarrhea, weight loss, fever, fatigue, anemia, loss of appetite and other symptoms. The brain-gut axis (BGA) refers to the bidirectional pathway formed between the central nervous system and the enteric nervous system. Studies have shown that the imbalance of the brain-gut axis plays an important role in the pathogenesis of Crohn's disease. Analysis of functional magnetic resonance imaging of the brain can non-invasively reflect the involvement of the nervous system in patients with Crohn's disease, thereby providing a basis for exploring the pathogenesis of Crohn's disease.

[0003] Traditional analysis of functional magnetic resonance imaging of the brain usually relies mainly on machine learning methods.

[0004] However, the intermediate process of traditional machine learning methods will be divided into multiple subtasks, making the overall analysis algorithm incoherent and often cannot be implemented through an end-to-end method in the form of input-output. The analysis results obtained are not necessarily the best results, thus affecting the overall detection accuracy. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides an end-to-end brain image data processing method and device based on deep learning. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] A deep learning-based end-to-end brain imaging data processing method, comprising:

[0007] Obtaining brain imaging data of a control group and a test group;

[0008] Performing feature extraction on the control group brain image data and the test group brain image data respectively, and obtaining a feature matrix to be trained and a feature matrix to be tested accordingly;

[0009] Build an end-to-end convolutional neural network classification model;

[0010] Training the end-to-end convolutional neural network classification model using the feature matrix to be trained;

[0011] The trained end-to-end convolutional neural network classification model is used to process the feature matrix to obtain a detection result.

[0012] In one embodiment of the present invention, the control group brain image data and the test group brain image data are both resting state functional magnetic resonance imaging data.

[0013] In one embodiment of the present invention, feature extraction is performed on the control group brain image data and the test group brain image data respectively, and a feature matrix to be trained and a feature matrix to be tested are obtained accordingly, including:

[0014] Calculating the functional connectivity features between pairs of brain regions in the brain image data of the control group according to the Pearson correlation coefficient, and converting the obtained functional connectivity features into a matrix form to obtain a feature matrix to be trained;

[0015] Correspondingly, the functional connectivity features between the brain region pairs in the brain image data of the test group are calculated according to the Pearson correlation coefficient, and the obtained functional connectivity features are converted into a matrix form to obtain a feature matrix to be tested.

[0016] In one embodiment of the present invention, the end-to-end convolutional neural network classification model includes an input layer, several hidden layers, a fully connected layer, a classification layer and an output layer in sequence; wherein the classification layer includes a softmax layer, a support vector machine and a random forest.

[0017] In one embodiment of the present invention, the plurality of hidden layers include a first hidden layer, a second hidden layer and a third hidden layer, and the first hidden layer is a "cross" shaped structure, and the second hidden layer and the third hidden layer are both "I" shaped structures.

[0018] In one embodiment of the present invention, training the end-to-end convolutional neural network classification model using the feature matrix to be trained includes:

[0019] Inputting the current feature matrix to be trained into the end-to-end convolutional neural network classification model through the input layer for forward propagation to obtain a model classification result;

[0020] Calculating a loss value according to the classification result of the model, and back-propagating the loss value to the input layer to update the weights of the network parameters of the end-to-end convolutional neural network classification model, and completing the training process of the current feature matrix to be trained;

[0021] The same method is used to process all feature matrices to be trained, thus completing one epoch of training;

[0022] Repeat the above steps until all feature matrices to be trained have completed a certain number of epoch training to complete the training of the end-to-end convolutional neural network classification model.

[0023] In one embodiment of the present invention, the current feature matrix to be trained is input into the end-to-end convolutional neural network classification model through the input layer for forward propagation to obtain a model classification result, including:

[0024] Inputting the current feature matrix to be trained into the end-to-end convolutional neural network classification model through the input layer, and processing it through several hidden layers and fully connected layers to obtain an output vector;

[0025] The output vector is input into the softmax layer, the support vector machine and the random forest respectively, and a convolution softmax classification result, a support vector machine classification result and a random forest classification result are obtained correspondingly;

[0026] The convolution softmax classification result, the support vector machine classification result and the random forest classification result are voted by using a voting method to obtain a model classification result.

[0027] Another embodiment of the present invention further provides an end-to-end brain image data processing device based on deep learning, comprising:

[0028] A data acquisition module, used to acquire brain image data of a control group and a test group;

[0029] A feature extraction module, used to extract features from the control group brain image data and the test group brain image data respectively, and obtain a feature matrix to be trained and a feature matrix to be tested accordingly;

[0030] Model building module, used to build an end-to-end convolutional neural network classification model;

[0031] A model training module, used to train the end-to-end convolutional neural network classification model using the feature matrix to be trained;

[0032] The data detection module is used to process the feature matrix to be detected by using the trained end-to-end convolutional neural network classification model to obtain a detection result.

[0033] Another embodiment of the present invention further provides an end-to-end brain image processing device based on deep learning, characterized in that it includes:

[0034] Memory, used to store computer programs;

[0035] The processor is used to implement the method steps provided in the above embodiment when executing the program stored in the memory.

[0036] Another embodiment of the present invention further provides a computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method steps provided in the above embodiment are implemented.

[0037] Beneficial effects of the present invention:

[0038] 1. The present invention integrates the entire analysis process into a whole by constructing an end-to-end convolutional neural network classification model, and implements the classification results based on the deep learning algorithm, so that the analysis algorithm is more coherent, and the output of the model directly reflects the input data expression information, thereby making the detection results more accurate;

[0039] 2. The end-to-end brain image data processing method based on deep learning provided by the present invention can be applied to brain image analysis equipment, which can not only provide more accurate inspection results, but also improve detection efficiency.

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of an end-to-end brain image data processing method based on deep learning provided by an embodiment of the present invention;

[0042] Figure 2 It is a schematic diagram of the structure of an end-to-end convolutional neural network classification model provided by an embodiment of the present invention;

[0043] Figure 3 It is a schematic diagram of the structure of an end-to-end brain image data processing device based on deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0045] Embodiment 1

[0046] See also Figure 1 , Figure 1 : is a flow chart of an end-to-end brain image data processing method based on deep learning provided by an embodiment of the present invention, which specifically includes the following steps:

[0047] S1: Obtain brain imaging data of the control group and the test group.

[0048] In this embodiment, the control group brain image data and the test group brain image data are both resting state functional magnetic resonance imaging data. Each sample of the control group brain image data has an abnormal or non-abnormal label. Specifically, in this embodiment, the abnormal sample can be the brain image data of a confirmed Crohn's disease individual, and the non-abnormal sample can be the brain image data of a healthy individual.

[0049] Specifically, this embodiment mainly obtains resting-state functional magnetic resonance brain imaging data of the control group and the test group through a functional magnetic resonance imaging device.

[0050] S2: extracting features from the control group brain image data and the test group brain image data respectively, and obtaining a feature matrix to be trained and a feature matrix to be tested accordingly.

[0051] In this embodiment, before extracting features from brain image data, preprocessing is required. The specific process is as follows:

[0052] First, the first five time series of functional MRI brain images were removed to avoid signal instability and allow the subjects to adapt to scanning noise. Then, the brain images were corrected for event history and head motion. When correcting the motion based on the record, the maximum translation of the tested individual in each direction was set to 2 mm, and the maximum angular motion of the tested individual during functional MRI imaging was set to 2°. Then, all functional images were standardized to the MNI152 template and smoothed with an isotropic Gaussian kernel (full width at half maximum [FWHM] = 4 mm) to remove the linear trend of the functional MRI brain images. A temporal filter was used to reduce the low-frequency drift and high-frequency physiological noise of the functional MRI brain images. White matter and cerebrospinal fluid were used as covariates for nuisance regression, and finally the brain image data of the control group and the test group were obtained.

[0053] Furthermore, feature extraction is performed on the preprocessed control group brain image data, including: calculating the functional connectivity features between brain region pairs in the control group brain image data according to the Pearson correlation coefficient, and converting the obtained functional connectivity features into a matrix form to obtain a feature matrix to be trained.

[0054] Firstly, the resting-state functional magnetic resonance imaging data of the preprocessed control group were calculated to obtain the functional connectivity (FC) between each pair of brain regions as the features to be trained.

[0055] Then, a matrix with the same width and height equal to the number of brain regions is taken, and the element value of each row represents the functional connectivity between a certain brain region and other brain regions. Each row is defined as the filling area of ​​the functional connectivity characteristic value of each brain region according to the order of brain regions;

[0056] Finally, the functional connectivity feature values ​​of each pair of brain regions are filled as the element values ​​of each point in the matrix according to the order of the brain regions to obtain the feature matrix to be trained.

[0057] Correspondingly, feature extraction of the preprocessed brain image data of the test group includes: calculating the functional connectivity features between brain region pairs in the brain image data of the test group according to the Pearson correlation coefficient, and converting the obtained functional connectivity features into a matrix form to obtain a feature matrix to be detected. The detailed process is the same as the feature extraction process of the control group brain image data.

[0058] S3: Build an end-to-end convolutional neural network classification model.

[0059] Specifically, see Figure 2 , Figure 2 It is a structural diagram of the end-to-end convolutional neural network classification model provided by an embodiment of the present invention, which includes: an input layer, several hidden layers, a fully connected layer, a classification layer and an output layer; wherein the classification layer includes a softmax layer, a support vector machine and a random forest.

[0060] Furthermore, the plurality of hidden layers include a first hidden layer, a second hidden layer and a third hidden layer, and the fully connected layer includes a fourth hidden layer, a fifth hidden layer and a sixth hidden layer.

[0061] This embodiment mainly improves the BrainNetCNN network to obtain an end-to-end convolutional neural network classification model (CrohnCNN) for processing brain image data. The specific process of the CrohnCNN model is as follows:

[0062] a) The convolution kernel of the first hidden layer is changed from a matrix to a "cross" shape, so that the receptive field of the convolution kernel changes from the surrounding neighborhood to the connection field of the brain region pair. In this embodiment, 32 convolution kernels are set, so that the feature matrix after the hidden layer processing increases to 32 times of the original.

[0063] b) The convolution kernel of the second hidden layer is changed from a matrix to a "one" shape, so that the receptive field of the convolution kernel changes from the connected domain of the brain area pair to the row convolution domain. The number of convolution kernels remains unchanged, so that the feature matrix after processing of this layer changes from a matrix to a vector.

[0064] c) The convolution kernel of the third hidden layer is kept in the shape of a straight line, and the direction is changed to convolution between different vectors, so that 32 vectors become 1 vector after being processed by the convolution kernel of this layer.

[0065] d) A fully connected layer is formed by the fourth hidden layer, the fifth hidden layer and the sixth hidden layer to process the output vector of the third hidden layer. The number of neurons in the fourth hidden layer, the fifth hidden layer and the sixth hidden layer gradually decreases.

[0066] e) Three classification modules are set to connect to the fully connected layer to classify the samples. The three classification modules are softmax layer, support vector machine classification layer, and random forest classification layer.

[0067] Specifically:

[0068] e1) Set the number of neurons in the softmax layer to 2, that is, use 2 neurons representing 0 and 1 respectively, where 0 represents an abnormal sample and 1 represents a non-abnormal sample. Use the output vector of the sixth hidden layer in the fully connected layer as the input of the softmax layer to obtain the convolution softmax classification result;

[0069] e2) using the output vector of the sixth hidden layer in the fully connected layer as the input of the support vector machine classification layer to obtain the support vector machine classification result;

[0070] e3) Use the output vector of the sixth hidden layer in the fully connected layer as the vector of the random forest classification layer as input to obtain the random forest classification result.

[0071] f) The above three classification modules are integrated, and the convolution softmax classification results, support vector machine classification results and random forest classification results are voted using the voting method. The label with the most votes is used as the final detection result of the CrohnCNN network.

[0072] S4: Train the end-to-end convolutional neural network classification model using the feature matrix to be trained.

[0073] 41) The current feature matrix to be trained is input into the end-to-end convolutional neural network classification model through the input layer for forward propagation to obtain the model classification result.

[0074] In this embodiment, the process in which the neural network model processes data from the input layer, through the hidden layer processing, until the output layer outputs the data is called forward propagation.

[0075] Specifically, step 41) includes:

[0076] 41-1) The current feature matrix to be trained is input into the end-to-end convolutional neural network classification model through the input layer, and is processed by several hidden layers and fully connected layers to obtain an output vector.

[0077] In this embodiment, the input data of the input layer of the neural network model is the feature matrix to be trained, which is data in the form of a two-dimensional matrix. Starting from this matrix, the convolution kernel is used to traverse all elements on the two-dimensional matrix, and then enters multiple hidden layers to process the input data.

[0078] First, the feature matrix is ​​increased to 32 times of its original size through the first hidden layer, and then the feature matrix is ​​converted from a matrix to a vector through the second hidden layer. The output data is then converted to a vector through the third hidden layer, and finally processed through the fully connected layer to output vector data.

[0079] 41-2) The output vector of the fully connected layer is input to the softmax layer, the support vector machine and the random forest respectively, and the convolution softmax classification result, the support vector machine classification result and the random forest classification result are obtained accordingly.

[0080] Specifically, the two neurons in the softmax layer, representing 0 and 1 respectively, each obtain a probability value of floating point number type, and the integer number represented by the neuron with a larger probability value is the classification result finally output by the convolutional softmax layer.

[0081] At the same time, support vector machine and random forest obtain corresponding classification results respectively.

[0082] 41-3) Use a voting method to vote on the convolution softmax classification results, support vector machine classification results and random forest classification results, and use the label with the most votes as the final detection result of CrohnCNN to obtain the final model classification result.

[0083] 42) Calculate the loss value according to the classification result of the model, and back-propagate the loss value to the input layer to update the weights of the network parameters of the end-to-end convolutional neural network classification model to complete the training process of the current feature matrix to be trained.

[0084] Specifically, the classification result obtained represents the label obtained by processing the sample through the neural network. After the difference between the label and the label of each sample itself is compared through the loss function, a loss value is obtained.

[0085] The loss value is set to the reverse direction of the forward propagation as the propagation direction, and the weights of the network parameters of all hidden layers are updated until the input layer. This process is a back propagation.

[0086] If the current feature matrix to be trained completes one forward propagation and one backward propagation, it indicates that the sample has completed one iteration.

[0087] 43) All feature matrices to be trained are processed in the same way, thus forming an epoch training.

[0088] When all feature matrices to be trained, that is, all samples, have completed one iteration, the end-to-end convolutional neural network classification model CrohnCNN is said to have completed one epoch of training.

[0089] 44) Repeat the above steps until all feature matrices to be trained have completed a certain number of epoch training to complete the training of the end-to-end convolutional neural network classification model.

[0090] Specifically, this embodiment completes the training process of the neural network model after completing 100 epochs using all the feature matrices to be trained, and obtains the trained end-to-end convolutional neural network classification model CrohnCNN.

[0091] S5: Use the trained end-to-end convolutional neural network classification model to process the feature matrix to be detected to obtain a detection result.

[0092] Specifically, the feature matrix to be detected is used as the input of the CrohnCNN model, and all feature matrices to be detected are input into the input layer of the CrohnCNN model. After forward propagation through the convolutional neural network, the output label 0 or 1 is obtained after processing, representing abnormal samples and non-abnormal samples respectively, and the result is used as the final detection result of the sample to be detected.

[0093] This embodiment integrates the entire analysis process into a whole by constructing an end-to-end convolutional neural network classification model, and implements the classification results based on the deep learning algorithm, so that the analysis algorithm is more coherent, and the output of the model directly reflects the information expressed by the input data, thereby making the detection results more accurate. The end-to-end brain image data processing method based on deep learning provided by the present invention is applied to brain image analysis equipment, which can not only provide more accurate inspection results, but also improve detection efficiency.

[0094] In addition, this embodiment also uses the GPU platform to accelerate the logarithm to provide accelerated computing for model construction and data detection, further improving the overall efficiency of the algorithm.

[0095] Embodiment 2

[0096] Based on the above-mentioned embodiment 1, this embodiment provides an end-to-end brain image data processing device based on deep learning. Figure 3 , Figure 3 : is a schematic diagram of the structure of an end-to-end brain image data processing device based on deep learning provided by an embodiment of the present invention, which includes:

[0097] A data acquisition module, used to acquire brain image data of a control group and a test group;

[0098] A feature extraction module, used to extract features from the control group brain image data and the test group brain image data respectively, and obtain a feature matrix to be trained and a feature matrix to be tested accordingly;

[0099] Model building module, used to build an end-to-end convolutional neural network classification model;

[0100] A model training module, used to train the end-to-end convolutional neural network classification model using the feature matrix to be trained;

[0101] The data detection module is used to process the feature matrix to be detected by using the trained end-to-end convolutional neural network classification model to obtain a detection result.

[0102] The end-to-end brain image data processing device based on deep learning provided in this embodiment can implement the end-to-end brain image data processing method based on deep learning provided in the above-mentioned embodiment 1, and the specific process will not be repeated here.

[0103] Embodiment 3

[0104] On the basis of the above-mentioned embodiment 1, this embodiment further provides an end-to-end brain image processing device based on deep learning, which can be applied to a brain image analysis device to process brain image data. It includes a processor and a memory, and the memory is used to store a computer program, and when the processor executes the computer program, the method steps described in the above-mentioned embodiment 1 can be implemented.

[0105] Embodiment 4

[0106] On the basis of the above-mentioned embodiment 1, this embodiment further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method steps described in the above-mentioned embodiment 1 are implemented.

[0107] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A deep learning-based end-to-end brain image data processing method, characterized in that: include: Obtaining brain imaging data of a control group and a test group; Performing feature extraction on the control group brain image data and the test group brain image data respectively, and obtaining a feature matrix to be trained and a feature matrix to be tested accordingly; Build an end-to-end convolutional neural network classification model; Training the end-to-end convolutional neural network classification model using the feature matrix to be trained; Using the trained end-to-end convolutional neural network classification model to process the feature matrix to be detected, to obtain a detection result; The end-to-end convolutional neural network classification model includes an input layer, a plurality of hidden layers, a fully connected layer, a classification layer and an output layer in sequence, wherein the classification layer includes a softmax layer, a support vector machine and a random forest, the plurality of hidden layers include a first hidden layer, a second hidden layer and a third hidden layer, and the first hidden layer is a "cross" structure, the second hidden layer and the third hidden layer are both "one" structures, the fully connected layer includes a fourth hidden layer, a fifth hidden layer and a sixth hidden layer, and the number of neurons in the fourth hidden layer, the fifth hidden layer and the sixth hidden layer gradually decreases; The step of training the end-to-end convolutional neural network classification model by using the feature matrix to be trained includes: Inputting the current feature matrix to be trained into the end-to-end convolutional neural network classification model through the input layer for forward propagation to obtain a model classification result; The step of inputting the current feature matrix to be trained into the end-to-end convolutional neural network classification model through the input layer for forward propagation to obtain a model classification result includes: Inputting the current feature matrix to be trained into the end-to-end convolutional neural network classification model through the input layer, and processing it through several hidden layers and fully connected layers to obtain an output vector; The output vector is input into the softmax layer, the support vector machine and the random forest respectively, and a convolution softmax classification result, a support vector machine classification result and a random forest classification result are obtained correspondingly; The convolution softmax classification result, the support vector machine classification result and the random forest classification result are voted by using a voting method to obtain a model classification result.

2. The end-to-end brain image data processing method based on deep learning according to claim 1, characterized in that: The brain image data of the control group and the brain image data of the test group are both resting-state functional magnetic resonance imaging data.

3. The end-to-end brain image data processing method based on deep learning according to claim 1, characterized in that: Feature extraction is performed on the control group brain image data and the test group brain image data respectively, and a feature matrix to be trained and a feature matrix to be tested are obtained accordingly, including: Calculating the functional connectivity features between pairs of brain regions in the brain image data of the control group according to the Pearson correlation coefficient, and converting the obtained functional connectivity features into a matrix form to obtain a feature matrix to be trained; Correspondingly, the functional connectivity features between the brain region pairs in the brain image data of the test group are calculated according to the Pearson correlation coefficient, and the obtained functional connectivity features are converted into a matrix form to obtain a feature matrix to be tested.

4. The end-to-end brain image data processing method based on deep learning according to claim 1, characterized in that: Training the end-to-end convolutional neural network classification model using the feature matrix to be trained also includes: Calculating a loss value according to the classification result of the model, and back-propagating the loss value to the input layer to update the weights of the network parameters of the end-to-end convolutional neural network classification model, and completing the training process of the current feature matrix to be trained; The same method is used to process all feature matrices to be trained, thus completing one epoch of training; Repeat the above steps until all feature matrices to be trained have completed a certain number of epoch training to complete the training of the end-to-end convolutional neural network classification model.

5. An end-to-end brain image data processing device based on deep learning, characterized in that: include: A data acquisition module, used to acquire brain image data of a control group and a test group; A feature extraction module, used to extract features from the control group brain image data and the test group brain image data respectively, and obtain a feature matrix to be trained and a feature matrix to be tested accordingly; A model building module, used to build an end-to-end convolutional neural network classification model, wherein the end-to-end convolutional neural network classification model includes an input layer, a plurality of hidden layers, a fully connected layer, a classification layer and an output layer in sequence, wherein the classification layer includes a softmax layer, a support vector machine and a random forest, the plurality of hidden layers include a first hidden layer, a second hidden layer and a third hidden layer, and the first hidden layer is a "cross" structure, the second hidden layer and the third hidden layer are both "one" structures, the fully connected layer includes a fourth hidden layer, a fifth hidden layer and a sixth hidden layer, and the number of neurons in the fourth hidden layer, the fifth hidden layer and the sixth hidden layer gradually decreases; A model training module, used to train the end-to-end convolutional neural network classification model using the feature matrix to be trained; A data detection module is used to process the feature matrix to be detected by using a trained end-to-end convolutional neural network classification model to obtain a detection result; The model training module is specifically used to input the current feature matrix to be trained into the end-to-end convolutional neural network classification model through the input layer, and obtain an output vector after being processed by several hidden layers and fully connected layers; the output vector is respectively input into the softmax layer, the support vector machine and the random forest to obtain the convolution softmax classification result, the support vector machine classification result and the random forest classification result accordingly; the convolution softmax classification result, the support vector machine classification result and the random forest classification result are voted by the voting method to obtain the model classification result.

6. An end-to-end brain image processing device based on deep learning, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 4 when executing a program stored in a memory.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 4 are implemented.

Images