Brain image classification methods and devices based on magnetic resonance imaging and deep learning
Through cascaded 3D_2D convolutional neural network and residual channel attention mechanism, the problem of model complexity and overfitting in MRI image classification is solved, and efficient brain area lesion feature extraction and classification is achieved, suitable for early Papa Diagnosis of Kingson's disease.
Patent Information
- Application Number
- CN202110995093.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-27
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-08-27
AI Technical Summary
The existing deep learning model based on MRI has problems with high memory usage and computational cost in brain region lesion classification, and it is difficult to effectively extract local lesion characteristics and is easy to overfit.
Using a brain image classification method based on magnetic resonance imaging and deep learning, MRI images are extracted through cascaded 3D_2D convolutional neural network model, low- and high-level features are extracted in combination with 3D and 2D branches, and residual channel attention is introduced. Force mechanism and depth can be separated and convolutional, and feature fusion is performed to improve classification accuracy.
It effectively reduces the complexity and calculation cost of the model, improves the classification accuracy and robustness, and makes the model run more efficiently in mobile terminals and embedded devices, and has high classification accuracy and stability.
Smart Images

Figure CN113705670B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer-aided information processing, and in particular to a brain image classification method and device based on magnetic resonance imaging and deep learning. Background Art
[0002] Parkinson's disease is the second most common neurological disease among the elderly population in the world. It is caused by the progressive loss of dopaminergic neurons in the substantia nigra pars compacta (SNc) of the midbrain. Its clinical manifestations mainly include tremors, rigidity, slow movements, difficulty walking, and activity disorders such as posture and balance disorders. In 2015, 6.2 million people were infected with Parkinson's disease worldwide, resulting in approximately 117,400 deaths. In addition, with the surge in the aging population worldwide, it has brought a huge economic burden to governments around the world. Therefore, early detection and monitoring of the disease are of great significance.
[0003] Magnetic resonance imaging scans of the brain provide anatomical details of subcortical structures. Texture and morphological analysis of different regions also provide objective imaging evidence for local lesions and heterogeneity of diseased areas. However, MRI is a three-dimensional structural imaging, and diagnosing diseases based on radiologists' reading of MRI images is often prone to errors. In recent years, machine learning methods have become a common tool for early diagnosis of brain diseases. Artificial features are extracted from MRI data to predict Parkinson's disease. However, due to its complex feature engineering and the high correlation between the subsequent classification or regression models, it often leads to poor learning performance of the model. Therefore, researchers pay more attention to data-driven feature representation without manual feature engineering that requires specific domain knowledge.
[0004] With the advancement of computer applications and intelligent systems, end-to-end medical image analysis using deep learning has shown advanced results. Deep learning architectures and more specifically convolutional neural networks can automatically extract effective information at different levels from high-dimensional imaging patterns. In addition, integrating feature extraction and model learning into a framework can further improve model processing performance. At present, some studies have been devoted to establishing MRI-based deep learning models for PD information processing, but there are still the following problems: (1) Using the entire three-dimensional MRI as input to build a 3D CNN model increases the difficulty of model optimization due to high memory usage and computational cost; (2) Using two-dimensional MRI as input to build a 2D CNN model does not provide sufficient information due to the absence of kernel sharing in the third dimension; (3) Brain lesions in early PD patients only occur in local areas, the target area is small, the noise information is large, and the deep convolutional network model is very easy to overfit. Therefore, building a deep learning network structure based on brain MRI imaging for the classification of brain lesions is still a challenging task. Summary of the invention
[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a brain image classification method and device based on magnetic resonance imaging and deep learning, to perform multi-scale and deep-level feature extraction on brain MRI images, thereby improving the classification accuracy and effectively reducing the complexity of the model.
[0006] The purpose of the present invention can be achieved by the following technical solutions:
[0007] A brain image classification method based on magnetic resonance imaging and deep learning comprises the following steps:
[0008] Acquire original MRI scan image data to be classified, perform preprocessing, and obtain a three-dimensional MRI image;
[0009] Using the three-dimensional MRI image as an input of a trained classification network model to obtain a final classification result;
[0010] Among them, the classification network model performs 3D to 2D feature dimension conversion on the three-dimensional MRI image along the axial plane, sagittal plane and coronal plane respectively, obtains corresponding initial classification results through the axial basis classifier, sagittal basis classifier and coronal basis classifier respectively, and then integrates and fuses multiple initial classification results to obtain the final classification result.
[0011] Furthermore, the axial position classifier, sagittal position classifier and coronal position classifier are all cascaded 3D_2D convolutional neural network models with the same structure.
[0012] Furthermore, the cascaded 3D_2D convolutional neural network model includes a 3D MRI input layer, a 3D convolution module, and a 3D branch and a 2D branch respectively connected to the 3D convolution module. The outputs of the 3D branch and the 2D branch are merged and then connected to the feature fusion layer and the final classification layer in sequence. The 3D convolution module is used to extract low-level features, and the 3D branch and the 2D branch are used to extract high-level features.
[0013] Furthermore, the 3D convolution modules are composed of several 3D convolution layers, batch normalization layers and activation layers, which are used to extract low-level features and three-dimensional context information.
[0014] Furthermore, the 3D branch includes a 3D convolution block, a 3D residual channel attention mechanism layer and a 3D global average pooling layer.
[0015] Furthermore, the 2D branch uses depthwise separable convolution to perform feature calculation along the channel direction, introduces a channel attention mechanism to calculate the weights of different channel features, and fuses information between channels through pixel-based 2D convolution.
[0016] Furthermore, the modality of the original MRI scan image data is T1-weighted MRI.
[0017] Furthermore, the preprocessing includes origin correction, N4 bias field correction, spatial normalization, skull stripping, filtering noise reduction and / or intensity normalization.
[0018] Furthermore, the spatial normalization process uses a symmetric normalization registration method to register all subjects' MRI scans to the MNI space.
[0019] Furthermore, the filtering and denoising process adopts a spatially adaptive non-local mean filter.
[0020] Furthermore, the intensity normalization process performs a centralization process on the image by removing the mean, and its mathematical expression is as follows:
[0021]
[0022] Among them, μ represents the image mean, σ represents the image standard deviation, and N represents the number of image pixels.
[0023] Furthermore, the mathematical expression of the cross entropy loss function used in the training of the classification network model is:
[0024]
[0025] Among them, q i Represents the probability that the neural network predicts that the sample belongs to each category, p i represents the true sample label, and ε is a hyperparameter.
[0026] Furthermore, the initial classification results are integrated and fused using a majority voting method.
[0027] The present invention also provides a computer-readable storage medium, characterized in that it includes one or more programs for execution by one or more processors of an electronic device, and the one or more programs include instructions for executing the brain image classification method based on magnetic resonance imaging and deep learning as described above.
[0028] The present invention also provides an electronic device, comprising:
[0029] one or more processors;
[0030] Memory; and
[0031] One or more programs stored in a memory, the one or more programs comprising instructions for executing the brain image classification method based on magnetic resonance imaging and deep learning as described above.
[0032] Furthermore, the electronic device may be a mobile terminal or an embedded device.
[0033] Compared with the prior art, the present invention has the following beneficial effects:
[0034] 1. The present invention performs information classification processing based on MRI images and deep learning. Through an end-to-end convolutional neural network, multi-view, deep-level feature extraction and feature fusion are performed on brain MRI images, thereby improving the automation and classification accuracy of the classification model.
[0035] 2. The present invention performs preprocessing such as N4 bias field correction, symmetric normalization registration method, spatial adaptive non-local mean filtering and intensity normalization on MRI images to reduce image noise and enhance local detail information to improve the quality of original MRI images.
[0036] 3. In the network model structure, the present invention designs a cascaded 3D_2D convolutional neural network model, which can enhance feature learning with fewer model parameters. Among them, the low-level 3D convolution can capture rich 3D context information, and the high-level feature extraction consists of two branches. The 3D branch ensures that the spatial features of the pixel points are not lost, and the 2D branch makes full use of the plane information of the pixel points. Through multi-scale feature fusion, the model has better classification accuracy and robustness.
[0037] 4. The present invention introduces a residual channel attention mechanism in the network model structure to enhance feature learning. In addition, the use of deep separable convolution operations can effectively reduce the computational complexity and parameter quantity of the entire model, avoid memory overflow and long training time due to an overly large network model, and improve the model calculation efficiency.
[0038] 5. The present invention provides an end-to-end classification model based on T1-weighted MRI imaging, which has high classification accuracy and stability. At the same time, the model has the advantages of smaller size, less computational effort, and light weight, which is conducive to the operation of the model in mobile terminals and embedded devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 It is a schematic diagram of the process of the present invention;
[0040] Figure 2 A schematic diagram of a cascaded 3D_2D convolutional neural network model in an embodiment;
[0041] Figure 3 Schematic diagram of a 3D convolution block structure in an embodiment;
[0042] Figure 4 A schematic diagram of a 3D residual channel attention mechanism block structure in an embodiment;
[0043] Figure 5Schematic diagram of the structure of a 2D convolution block in an embodiment. DETAILED DESCRIPTION
[0044] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0045] Example 1
[0046] like Figure 1 As shown, this embodiment provides a brain image classification method based on magnetic resonance imaging and deep learning, including the following steps: obtaining the original MRI scan image data to be classified, performing preprocessing, and obtaining a three-dimensional MRI image; using the three-dimensional MRI image as the input of a trained classification network model to obtain the final classification result. Among them, the classification network model converts the feature dimensions of the three-dimensional MRI image from 3D to 2D along the axial plane, sagittal plane, and coronal plane, respectively, obtains the corresponding initial classification results through the axial position based classifier, sagittal position based classifier, and coronal position based classifier, respectively, and then integrates and fuses multiple initial classification results to obtain the final classification result. The axial position based classifier, sagittal position based classifier, and coronal position based classifier are all cascaded 3D_2D convolutional neural network models with the same structure.
[0047] In this embodiment, the above brain lesion classification method is applied to the classification of brain images of early Parkinson's disease and brain images of healthy people. The construction and training process of the classification network model of this embodiment includes the following steps:
[0048] Step 1: Data preparation and preprocessing
[0049] 1. MRI image data: We obtained original T1-weighted MRI image data of 430 subjects, including 229 Parkinson's disease patients and 201 healthy subjects matched in age and gender ratio.
[0050] 2. Preprocessing: The MRI images are preprocessed as follows:
[0051] (1) AC-PC pre-joint point correction;
[0052] (2) Using the N4 bias field correction method to balance the inhomogeneity in MRI scans;
[0053] (3) MRI scans of all subjects were registered to MNI space using the symmetric normalized registration method;
[0054] (4) skull stripping using the binary MNI spatial brain template;
[0055] (5) Using a spatially adaptive non-local mean filter to remove noise from the image while preserving the edges;
[0056] (6) The image is de-meaned and centralized for intensity normalization, rescaling the voxel values to zero mean and unit standard deviation, making it easier to obtain generalization effects after training. The mathematical expression is as follows:
[0057]
[0058] Among them, μ represents the image mean, σ represents the image standard deviation, and N represents the number of image pixels.
[0059] 3. Data division: All pre-processed MRI images of the subjects are divided into training set, validation set and test set according to categories. The five-fold cross-validation method is used to divide the training set, validation set and test set.
[0060] 4. Data enhancement: The online data enhancement method is used to perform data enhancement processing on the training set data by randomly adding noise, and no data enhancement processing is performed on the validation set and the test set.
[0061] Step 2: Network model design and training
[0062] The classification network model of this embodiment includes an axial basis classifier, a sagittal basis classifier, a coronal basis classifier, and an integrated fusion unit. The axial basis classifier, the sagittal basis classifier, and the coronal basis classifier are base classifier models based on different single-axis views, and all use the same cascade 3D_2D convolutional neural network model.
[0063] like Figure 2 As shown, the cascaded 3D_2D convolutional neural network model of this embodiment includes a 3D MRI input layer, two 3D convolution modules, and then divided into two branches, 3D branch and 2D branch, for deep feature extraction. Finally, the deep features obtained from each branch are fused through the feature fusion layer and input to the final classification layer. Among them, the 3D convolution module part consists of several 3D convolution layers, batch normalization layers and activation layers, which are used to extract low-level features and three-dimensional context information, such as Figure 3 As shown. Secondly, the 3D branch consists of a 3D convolution block, a 3D residual channel attention mechanism layer, and a 3D global average pooling layer to extract high-level features and ensure that the spatial information of the pixels is not lost, as shown Figure 4 In the 2D branch, a depth-separable convolution with a kernel size of 3×3 is used to extract channel-based feature maps, and then a channel attention mechanism is introduced to calculate the weights of different channel features to enhance the channel feature expression capability that is beneficial to the classification results. Finally, a pixel-based 2D convolution with a kernel size of 1×1 is used to fuse information between channels, as shown in Figure 5 shown.
[0064] Add l to the convolutional layer 2 Kernel weight regularization, the regularization coefficient is set to l 2 , the number of batch samples is 16, and the initial learning rate is set to l r , the number of training iterations is 300, the momentum stochastic gradient descent optimizer is used to optimize the model parameters, and the network connection weights are trained by gradient back propagation to minimize the loss function. The loss function used is the improved binary cross entropy loss function, and its mathematical expression is:
[0065]
[0066] Among them, q i Represents the probability that the neural network predicts that the sample belongs to each category, p i represents the true sample label, and ε is a small hyperparameter.
[0067] During the model training process, the Bayesian optimization algorithm is used to optimize the hyperparameters. The hyperparameter regularization coefficient l is set respectively 2 The initial optimization range is {1e -6 ,1e -2}, initial learning rate l r The optimization range is {1e -8 ,1e -2 By observing the classification performance of the model on the validation set, the hyperparameter optimization range is further narrowed until the lowest loss and highest classification accuracy are achieved on the validation set. The current hyperparameter value is then selected as the optimal hyperparameter for the model.
[0068] The prediction results of the three base classifiers based on the single-axis view are integrated to obtain the final classification result. Here, the majority voting method is used to integrate the three predictions based on the single-axis view, and finally the prediction label integrated by the base classifiers of the three-axis view is obtained. The model fusion process is as follows:
[0069] (1) The prediction results based on the three single-axis views are recorded as: {p 1 ,p 2 ,p 3};
[0070] (2) Accumulate and sum the prediction results based on the three single-axis views: p = p 1 +p 2 +p 3 ;
[0071] (3) Determine the predicted label of the final model based on the p-value:
[0072] Step 3: Model Evaluation
[0073] In this experiment, the evaluation indicators of the test results include: Accuracy (ACC), Sensitivity (Sen), Specificity (Spe), F 1 The score and the area under the receiver operating characteristic curve (AUC) value are calculated as follows:
[0074]
[0075]
[0076]
[0077]
[0078] Among them, TP represents the number of samples in which patients are correctly classified; FP represents the number of samples in which patients are incorrectly classified; TN represents the number of samples in the healthy control group that are correctly classified; and FN represents the number of samples in the healthy control group that are incorrectly classified.
[0079] This experiment first trains the base classifier models based on the single-axis view and tunes the model parameters according to the performance of the validation set; secondly, the trained base classifier models of the three-axis view are operated by the majority voting mechanism to obtain the prediction results of the final model on the test set; finally, the average classification performance of the model is calculated based on the five-fold cross validation. The experimental results are shown in Table 1.
[0080] Table 1 Average classification results on the test set
[0081]
[0082] The present invention designs an experiment to study the classification of early Parkinson's disease based on T1-weighted MRI images, and constructs an end-to-end Parkinson's disease computer-aided diagnosis model based on deep learning. The proposed early Parkinson's disease classification model is obtained by integrating the base classification models based on three single-axis views, and the axial base classifier, sagittal base classifier and coronal base classifier are all a cascade 3D_2D convolutional neural network model with the same structure. First, the three-dimensional MRI image of the brain is used as input, and then the feature dimension conversion from 3D feature map to 2D feature map is performed along the axial plane, sagittal plane and coronal plane respectively, so as to construct three base classifier models based on different single-axis views, and finally the majority voting method is used to integrate the three predictions based on the single-axis view to obtain the final model prediction result. Compared with the base model based on the single-axis view, the present invention can achieve an average classification accuracy of 89.65% on the basis of model fusion, and has a good early Parkinson's disease recognition effect.
[0083] In this embodiment, the classification network model constructed and trained as above is applied to the classification of brain images of early Parkinson's disease and healthy people. The specific process is: the original MRI scan image data to be classified is obtained, and the preprocessing as described above is performed, and the data is input into the classification network model. Three initial classification results are obtained by the axial basis classifier, the sagittal basis classifier and the coronal basis classifier. The three initial classification results are fused by using the voting mechanism to obtain the final classification result.
[0084] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.
[0085] Example 2
[0086] This embodiment provides an electronic device, including one or more processors, a memory, and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the brain image classification method based on magnetic resonance imaging and deep learning as described in Example 1.
[0087] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.
Claims
1. A brain image classification method based on magnetic resonance imaging and deep learning, It is characterized in that The following steps are involved: Acquire original MRI scan image data to be classified, perform preprocessing, and obtain a three-dimensional MRI image, wherein the modality of the original MRI scan image data is T1-weighted MRI, and the preprocessing includes origin correction, N4 bias field correction, spatial normalization, skull stripping, filtering noise reduction, and / or intensity normalization; Using the three-dimensional MRI image as an input of a trained classification network model to obtain a final classification result; The classification network model performs 3D to 2D feature dimension conversion on the three-dimensional MRI image along the axial plane, sagittal plane and coronal plane, respectively, obtains corresponding initial classification results through the axial position-based classifier, the sagittal position-based classifier and the coronal position-based classifier, respectively, and then integrates and fuses multiple initial classification results to obtain the final classification result; The axial position classifier, sagittal position classifier and coronal position classifier are all cascaded 3D_2D convolutional neural network models with the same structure; The cascaded 3D_2D convolutional neural network model includes a 3D MRI input layer, a 3D convolution module, and a 3D branch and a 2D branch respectively connected to the 3D convolution module, wherein the outputs of the 3D branch and the 2D branch are merged and then sequentially connected to a feature fusion layer and a final classification layer, wherein the 3D convolution module is used to extract low-level features, and the 3D branch and the 2D branch are used to extract high-level features; The 3D branch includes a 3D convolution block, a 3D residual channel attention mechanism layer, and a 3D global average pooling layer; The 2D branch uses depthwise separable convolution to perform feature calculation along the channel direction, introduces a channel attention mechanism to calculate the weights of different channel features, and fuses information between channels through pixel-based 2D convolution.
2. The brain image classification method based on magnetic resonance imaging and deep learning according to claim 1, It is characterized in that The initial classification results are integrated and fused using the majority voting method.
3. A computer-readable storage medium, It is characterized in that It comprises one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the brain image classification method based on magnetic resonance imaging and deep learning as described in any one of claims 1-2.
4. An electronic device, It is characterized in that include: one or more processors; Memory; and One or more programs stored in a memory, wherein the one or more programs include instructions for executing the brain image classification method based on magnetic resonance imaging and deep learning as described in any one of claims 1-2.
Citation Information
Patent Citations
Medical image feature extraction and classification method under cascaded neural network
CN112215291A
Feature brain region positioning method of magnetic resonance imaging data based on deep learning
CN112465905A
Database management for laparoscopic surgery
WO2017098503A1
