Mri image classification method and system based on feature extraction and attention mechanism

Abstract

The application belongs to the technical field of image classification, and provides an MRI image classification method and system based on feature extraction and attention mechanism, which comprises the following steps: acquiring MRI image data and performing pretreatment; and performing image classification by using a pre-trained bilinear convolutional neural network model; wherein the bilinear convolutional neural network model comprises a first network model, a second network model and a classifier; the first network model and the second network model have the same structure and specifically comprise a convolutional layer, a first maximum pooling layer, a residual block combination unit and a second maximum pooling layer; the residual block combination unit is a multi-branch convolutional residual block; an attention mechanism is added between the residual block combination unit and the second maximum pooling layer, and weights are added to the channel attention of the MRI image features processed by the residual block combination unit; and the application solves the problems of few medical image data samples and weak generalization ability by adding small residual blocks in the residual unit structure.

Description

Technical Field

[0001] This invention belongs to the field of image classification technology, specifically relating to an MRI image classification method and system based on feature extraction and attention mechanisms. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Alzheimer's disease (AD) is a progressive neurodegenerative disorder characterized by insidious onset and continuous decline in brain function. Its primary manifestation is a rapid decline in cognitive and memory functions. The generally accepted pathogenesis involves neuronal damage and synaptic loss caused by intracellular neurofibrillary tangles and extracellular β-amyloid protein deposition. In AD diagnostic classification, numerous images can visualize the brain structure of patients. Among these, magnetic resonance imaging (MRI) plays a crucial role in AD diagnosis. It is a widely available and non-invasive biomarker and a primary medium for demonstrating the progression of AD. Using MRI to assess atrophy in the temporal lobe and hippocampus regions improves the accuracy of AD diagnosis.

[0004] Because medical images are characterized by high noise, low resolution, and small datasets, and because image prediction, diagnosis, and classification require a focus on changes in local regions of the image, traditional learning methods that rely on manual feature extraction are no longer sufficient to meet the requirements of image diagnosis and classification.

[0005] Currently, machine learning shows good performance in the diagnosis and classification of Alzheimer's disease (AD). Canisius analyzed MRI image data using the random forest algorithm, which can extract low-dimensional features and representations from high-dimensional MRI data. Jha et al. used two-dimensional discrete wavelet (2D-DWT) for feature extraction with filtering and denoising, followed by dimensionality reduction using probabilistic principal component analysis (PPCA), and then employed a random subspace ensemble classifier (RSE) based on the K-nearest neighbor (KNN) algorithm to perform binary classification of diseased brain images into pathological and normal types. With the emergence and development of deep learning, compared to traditional image classification methods, it eliminates the need for manual feature description and extraction of target images, solving the shortcomings of traditional detection and classification. Instead, it autonomously learns features from training samples through neural networks, extracting higher-dimensional and abstract features that are closely related to the classifier, effectively solving the problems of manual feature extraction and classifier selection. Arezou et al. demonstrated different stages of Alzheimer's disease by combining feature fusion (FF) with sparse denoising autoencoders (SDAE). They first extracted high-level features using stacked autoencoders, then used multinomial logistic regression, a fusion of high- and low-level features, to classify the data. Subsequently, Sui proposed a convolutional neural network based on a combination of missing-correction linear units (L-RELU) and max pooling for AD diagnosis and classification. This model used an eight-layer convolutional and pooling structure, which improved classification accuracy, but suffered from excessive hyperparameters and high overall time complexity.

[0006] Based on the techniques described above, the proposed method represents a significant advancement and exhibits strong generalization ability. However, its drawbacks include high time complexity due to the use of numerous hyperparameters, and the inherent noise and low resolution of Alzheimer's disease data, resulting in poor interpretability. Consequently, challenges have arisen in diagnostic classification: the amount of data available for medical images is limited, necessitating data augmentation to expand the data and prevent overfitting; medical images differ from natural images, as AD images are formed using methods such as MRI, requiring similarity and fine-structure analysis for diagnostic classification, necessitating feature extraction based on the image data structure to find the required feature representations; and the deep learning network model framework, acting as a black box, cannot accurately explain the experimental process and significance, failing to address the interpretability issues in medical image classification. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes an MRI image classification method and system based on feature extraction and attention mechanisms. This invention extracts multi-scale features by increasing the cardinality, adds small residual blocks to the residual unit structure, enhances the connection between local and global blocks, and incorporates a CABW structure before aggregation after extraction for refined information allocation and processing. By using an SVM classifier, it solves the problems of limited medical image data samples and weak generalization ability. This method is applicable to binary classification experiments in AD, NC, and MCI, achieving better classification accuracy and robustness.

[0008] According to some embodiments, the first aspect of the present invention provides an MRI image classification method based on feature extraction and attention mechanisms, employing the following technical solution:

[0009] MRI image classification methods based on feature extraction and attention mechanisms include:

[0010] Acquire MRI image data and perform preprocessing;

[0011] Image classification is performed using a pre-trained bilinear convolutional neural network model.

[0012] The bilinear convolutional neural network model includes a first network model, a second network model, and a classifier; the first network model and the second network model have the same structure, specifically including:

[0013] Convolutional layers, max pooling layers, residual block combinatorial units, and pooling layers;

[0014] The residual block combination unit is a multi-branch convolutional residual block;

[0015] An attention mechanism is added between the residual block combination unit and the pooling layer to add channel attention weights to the MRI image features processed by the residual block combination unit.

[0016] Furthermore, the acquisition and preprocessing of MRI image data includes:

[0017] Acquire MRI image data;

[0018] Two-dimensional MRI planar images are obtained based on MRI image data;

[0019] Two-dimensional MRI planar images are segmented to obtain gray matter images, white matter images, and cerebrospinal fluid images;

[0020] Spatial normalization and smoothing were performed on gray matter images, white matter images, and cerebrospinal fluid images to obtain preprocessed MRI images.

[0021] Furthermore, the image classification using a pre-trained bilinear convolutional neural network model includes:

[0022] The preprocessed MRI images are divided into a first MRI image and a second MRI image;

[0023] A first network extracts a first feature from a first MRI image; a second network extracts a second feature from a second MRI image;

[0024] The first and second features are fused by dot product to obtain the final MRI image features;

[0025] The final MRI image features are classified using a classifier to obtain the final MRI image classification result.

[0026] Furthermore, the first network extracts a first feature from the first MRI image, specifically including:

[0027] Feature extraction was performed on the first MRI image using convolutional layers;

[0028] The extracted first MRI image features are reduced in dimensionality using a max pooling layer;

[0029] For the first MRI image after dimensionality reduction, multi-scale features of the first MRI image are obtained by using residual block combination units;

[0030] By using a channel attention mechanism to add weights to the multi-scale features of the first MRI image, multi-scale features of the first MRI image with attention weights are obtained.

[0031] The first feature is obtained by using a pooling layer to reduce the dimensionality of the multi-scale features of the first MRI image with attention weights.

[0032] Furthermore, the second network extracts a second feature from the second MRI image, specifically including:

[0033] Feature extraction of the second MRI image was performed using convolutional layers;

[0034] The extracted second MRI image features are reduced in dimensionality using a max pooling layer.

[0035] For the dimension-reduced second MRI image, multi-scale features of the second MRI image are obtained by using residual block combination units;

[0036] By using a channel attention mechanism to add weights to the multi-scale features of the second MRI image, multi-scale features of the second MRI image with attention weights are obtained.

[0037] The second feature is obtained by using a pooling layer to reduce the dimensionality of the multi-scale features of the second MRI image with attention weights.

[0038] Furthermore, for the dimension-reduced first MRI image, multi-scale features of the first MRI image are obtained using residual block combination units, including:

[0039] The feature map of the first MRI image after 1x1 division is divided into s blocks according to the number of channels, and each block is x in size. i ;

[0040] The first convolution operation begins with the first block x1;

[0041] As i increases, the i-th block is added to the (i-1)-th block after convolution, and then the i-th convolution operation is performed.

[0042] By concatenating each convolutional segment, the multi-scale features of the first MRI image are obtained.

[0043] Furthermore, for the dimension-reduced second MRI image, multi-scale features of the second MRI image are obtained using residual block combination units, including:

[0044] For the dimension-reduced second MRI image, multi-scale features of the second MRI image are obtained using residual block combination units, including:

[0045] The feature map of the second MRI image after 1x1 is divided into s blocks according to the number of channels, and each block is x in size. i ;

[0046] The first convolution operation begins with the first block x1;

[0047] As i increases, the i-th block is added to the (i-1)-th block after convolution, and then the i-th convolution operation is performed.

[0048] By concatenating each convolutional segment, the multi-scale features of the second MRI image are obtained.

[0049] According to some embodiments, a second aspect of the present invention provides an MRI image classification system based on feature extraction and attention mechanisms, employing the following technical solution:

[0050] MRI image classification systems based on feature extraction and attention mechanisms include:

[0051] The image processing module is configured to acquire MRI image data and perform preprocessing.

[0052] The image classification module is configured to perform image classification using a pre-trained bilinear convolutional neural network model.

[0053] The bilinear convolutional neural network model includes a first network model, a second network model, and a classifier; the first network model and the second network model have the same structure, specifically including:

[0054] Convolutional layer, first max pooling layer, residual block combination unit and second max pooling layer;

[0055] The residual block combination unit is a multi-branch convolutional residual block;

[0056] An attention mechanism is added between the residual block combination unit and the second max pooling layer to add channel attention weights to the MRI image features processed by the residual block combination unit.

[0057] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0058] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps in the MRI image classification method based on feature extraction and attention mechanisms as described in the first aspect above.

[0059] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0060] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the MRI image classification method based on feature extraction and attention mechanisms as described in the first aspect above.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] To improve the accuracy and robustness of Alzheimer's disease MRI image classification by extracting discriminative features from images, this invention proposes an improved network model combining residual block feature extraction and an attention mechanism, based on a bilinear convolutional neural network (BCNN) model. This method replaces the original VGGNet model in the bilinear model with an improved ResNet model, addressing the challenges of SGD optimization, poor learning performance, and network degradation as network complexity increases. It extracts multi-scale features and increases the receptive field of each layer without increasing complexity, while utilizing an end-to-end model to enhance interpretability. During feature extraction, an attention mechanism is used to select weights for the acquired features, leveraging nonlinear relationships on the channels to obtain more expressive feature vectors. Spatial attention is then added to further enhance image features with higher weights. An SVM Gaussian kernel classifier is used, which is advantageous for small to medium-sized data samples, nonlinear and high-dimensional classification, and solves the problem of poor generalization ability caused by softmax in previous models. Experimental results show that, compared with other models in AD diagnosis classification, the improved extraction method combined with the attention mechanism exhibits better robustness and higher classification accuracy. Attached Figure Description

[0063] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0064] Figure 1 This is a diagram of the bilinear convolutional neural network model structure in the MRI image classification method based on feature extraction and attention mechanisms described in Embodiment 1 of the present invention.

[0065] Figure 2 This is a schematic diagram of the process of segmenting the original MRI image as described in Embodiment 1 of the present invention;

[0066] Figure 3 This is a schematic diagram of the preprocessing process described in Embodiment 1 of the present invention;

[0067] Figure 4 This is a structural diagram of the residual combination unit as described in Embodiment 1 of the present invention. Detailed Implementation

[0068] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0069] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0070] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0071] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0072] Example 1

[0073] like Figures 1-4 As shown, this embodiment provides an MRI image classification method based on feature extraction and attention mechanisms. The method includes the following steps:

[0074] Acquire MRI image data and perform preprocessing;

[0075] Image classification is performed using a pre-trained bilinear convolutional neural network model.

[0076] The bilinear convolutional neural network model includes a first network model, a second network model, and a classifier; the first network model and the second network model have the same structure, specifically including:

[0077] Convolutional layers, max pooling layers, residual block combinatorial units, and pooling layers;

[0078] The residual block combination unit is a multi-branch convolutional residual block;

[0079] An attention mechanism is added between the residual block combination unit and the pooling layer to add channel attention weights to the MRI image features processed by the residual block combination unit.

[0080] like Figure 1 As shown, the specific implementation steps of the method described in this embodiment are as follows:

[0081] Step S1: The dataset images need to be preprocessed first. The images are viewed and processed using the tools Matlab and SPM12 to obtain the 2D image plane required for the model.

[0082] Step S2: The first improvement is made to the residual blocks in ResNet by increasing the "cardinality" to 32 parallel channels, changing the single branch to a multi-branch approach, and reducing the size of the preprocessed image features to 224x224 with 256 channels. The L1 layer reduces the kernel size from 1x1x128 to 1x1x4, the L2 layer reduces the kernel size from 3x3x128 to 3x3x4 to achieve dimensionality reduction, and the L3 layer reduces the kernel size to 1x1x256. The final matrix is ​​obtained through aggregation. These three layers form one channel, thus expanding the width. The second improvement is to refine the dimensionality reduction convolution. The features before dimensionality reduction are divided into four equal parts, and the divided features are then processed using the K() convolution method with a 3x3 kernel. i =K()+x i-1 The idea is to derive each feature and then concatenate them to obtain the final matrix features.

[0083] Step S3: Apply the improved residual blocks from Step S2 to ResNet, replacing the VGGNet feature extraction model in the bilinear model. Using the bilinear basic network model, extract features from local areas A and B of the image separately to obtain feature representations. During feature extraction, to highlight the important parts of the image features we are extracting, an attention mechanism is added. Using the CABW structure, channel attention is applied to each group of residual blocks to add weights and obtain important features. Finally, spatial attention is used before dot product fusion to obtain the feature data with the required weights.

[0084] Step S4: Finally, SVM is used to perform supervised classification of the data, using the Gaussian kernel function.

[0085] Step S5: Train the model using the weighted cross-entropy loss function to achieve accurate classification and recognition on the dataset.

[0086] Specifically, step S1, the process of preprocessing the acquired image, includes:

[0087] The image dataset used in this embodiment is medical image data. This data is stored in 3D planar format (.nii). A 2D planar image is obtained from the 3D image using a longitudinal method to view the image slice content. All magnetic resonance imaging (MRI) data meeting the experimental requirements are extracted from the dataset. Since the images are stored as files and are three-dimensional, Matlab and SPM12 are used to preprocess the images to obtain the necessary experimental images. First, format conversion is required for experimental convenience; for example, if the downloaded format is DICOW, it is converted to NIfTI format. Second, since the original images in ADNI contain redundant parts such as the skull and cerebral cortex, the brain image needs to be segmented into gray matter, white matter, and cerebrospinal fluid using Cat12 in SPM12 to reduce the computational load on the model for image training. Specific operation results are as follows: Figure 2 As shown. Finally, spatial normalization and smoothing are performed on the segmented images to achieve image consistency standards, reduce image noise interference, and ensure that interference caused by differences in anatomical structures after segmentation is eliminated as much as possible, ultimately achieving the image data required for the experiment. Taking gray matter as an example, the specific operation is as follows: Figure 3 As shown.

[0088] In step S2, the residual block assembling unit adopts the improved ResNet model, specifically including:

[0089] The underlying network used in this embodiment is the ResNet model. Based on the ResNet model, an improvement to the ResNet residual block is proposed. It adopts the ResNet repeating layer strategy and implements it in a simple and scalable way using the split-transform-merge strategy. One module performs a set of transformations, each based on low-dimensional embeddings. Its output is aggregated by summation. During transformation, it is built by building blocks, and transformations with the same extension are aggregated in the blocks.

[0090] To improve the representational power of the model, we changed the filter from a single branch to a multi-branch filter. By adding the concept of 'radix' to the improved 3x3 residual block, we obtained a residual block combination unit structure. The preprocessed image features were 224x224 in size. After passing through a 7x7 convolutional layer and a 3x3 max pooling layer, we obtained a size of 256x224x224. First, we split the original residual block into 32 groups based on a radix of 32, and then obtained 4x224x224 features through 4 1x1 convolutional kernels. Then, we obtained features with 4 channels through 3x3 convolutional kernels. This transformation has the effect of downsampling the image data to achieve dimensionality reduction. We then convolved the dimensionality-reduced features to obtain 256x224x244 features, and finally achieved the aggregation effect. During model aggregation, we replaced the network structure within the neurons with functions from the original model, thereby increasing the depth dimension, as shown in formula (1).

[0091]

[0092] In the formula, Ti(x) is the function used before aggregation, y = {X1, X2, X3, ..., X...} n |n is the number of base numbers}, X i x is from 1 to i i The sum of these values, where C is the cardinality of the transformation, and F(y) is the result after aggregation. This feature information fusion method not only increases the receptive field size of each layer but also obtains multi-scale features.

[0093] Building upon this, we further improved the dimensionality-reducing convolution within residual blocks with added cardinality by replacing the existing single structure within the residual with smaller, segmented residual blocks, such as... Figure 3 As shown, the main improvement is to the 3x3 residual block, performing fine-grained multi-scale parallel processing. First, the feature map after passing through 1x1 is divided into s blocks according to the number of channels. We choose the case where s=4, and each small block is x i Size, x after dividing i Each feature undergoes a 3x3 convolution, and each 3x3 convolution operator can receive feature information from feature xi. Let K() represent the convolution process. Initially, x1 undergoes a direct convolution operation. As i increases, x... i With K i-1 Add the results together and then send them to K. i In the process of (), the loop is performed sequentially, and finally these segmented output blocks are concatenated. According to the improvement process, formula (2) shows the output result after the block, which divides the segmented features into 4 blocks according to the above improvement, where the convolution of the first block after segmentation is omitted, and the second block performs the convolution function K. i(), in subsequent blocks, we add the output of the previous block to the current block after convolution and output the result. The advantage is that it is beneficial for the extraction of local and global information and the correlation between them, and the replacement effect achieves the accuracy of feature information.

[0094]

[0095] Where, x i K is the size of the residual block after segmentation, and K() represents the convolution process.

[0096] We replace the VGGNet feature extraction part of the bilinear CNN model with an improved ResNet network model. The bilinear model consists of four parts, namely M... a M b P, C, where M a and M b It is the backbone network model for feature extraction, where P is the pooling layer and C is the classification function. In the feature extraction process, image data d (d∈D) and image position l (l∈L) are selected, and their output is a feature matrix of size m*n. The bilinear model calculation process is shown in formula (3):

[0097] bilinear(d,l,M a M b ) = M a (d,l) T M b (d,l) (3)

[0098] In this model, since information fusion requires dot product operations, the feature matrices obtained from paths a and b must be of the same size. The pooling layer is used to combine M... a and M b The dot product is aggregated to obtain G(d), where L refers to the sum of positions in the image, as shown in formula (4):

[0099]

[0100] The entire backbone network structure has been built. This method can not only extract medical images on multi-scale features, but also combine the correlation between global and local information.

[0101] Building upon this foundation, an attention mechanism was added. The overall bilinear model structure captures the correlation of features by performing dot product aggregation on the two sets of feature data, and by utilizing the nonlinear relationship on the channels in our improved ResNet, more expressive feature vectors are obtained. However, the model does not emphasize the weights of the target locations we need in the overall extraction. To address this deficiency, we adopted a CBAW structure, adding a channel attention mechanism after the residual blocks of the first and second networks, and a spatial attention mechanism before pooling. By learning features through CBAW, higher weights are assigned to the image, enhancing the expressive power of the bilinear model. The attention mechanism is then incorporated into the improved bilinear model as follows: Figure 4 As shown. Here, X' is the input matrix of the residual block, assuming m = [m1, m2, m3, ..., m...]. c The convolution kernel is formed after the residual blocks are aggregated, and the output feature map is T = [t1, t2, t3, ..., t]. c [,] Channel attention mechanism is applied to T. First, for t i Global average pooling and global max pooling are used to effectively learn information and location, as shown in formulas (5) and (6), where Q and H are the feature dimensions. Then, activation functions are used to calculate channel weights, as shown in formula (7).

[0102]

[0103] z c2 =max(t) c (i,j))(i∈[1,2,3,...,Q],j∈[1,2,3,...,H]) (6)

[0104] s=σ(MLP( zc1 )+,MLP( zc2 ))=σ(W2Φ(W 1zc1 )+W2Φ(W 1zc2 (7)

[0105] Where Φ() is the ReLU activation function, W1∈RcWlxc and W2 are the weights of the FC layer. Then, the obtained channel weights are multiplied by the input feature map T to obtain a weighted feature map with the same size, as shown in formula (8).

[0106] x c =G scale (t c ,s c ) = s c t c (8)

[0107] Based on the structure of the hybrid attention mechanism, we place spatial attention after channel attention, which is a supplement to channel attention. We use the weights to apply spatial attention to the extracted image.

[0108] The attention mechanism proposed in this paper enables the improved network model to have stronger local extraction capabilities. At the same time, it combines the correlation between global and local information to obtain richer information, which improves the classification of medical images.

[0109] The loss function in the classification model is the weighted cross-entropy loss function, which is used to evaluate the degree of difference between the model's predicted output and the ground truth. It can intuitively reflect the training effect of the model. Generally, the smaller the loss, the closer the predicted output is to the ground truth, and the better the model's performance. The Dropout_ratio is set to 0.5, the batch size is set to 100, the epoch is set to 85, and the momentum is set to 0.1. The weighted cross-entropy loss function in the binary classification calculation process is shown in formula (9):

[0110]

[0111] In formula (9), y i Let represent the label of sample image i, with 1 for positive class and 0 for negative class; pi represents the probability of being predicted as positive; β∈[0,1] is used to control the degree of penalty imposed by the model on FNWFP: when β is set less than 0.5, due to the penalty imposed on (1-y i )˙log(1-p i The weights of FP are greater, so the penalty for FP is greater than that for FN, and vice versa. In order to optimize the classification model, the gradient descent algorithm Adam is used as the backpropagation algorithm, and alpha is set to 0.001, beta1 to 0.9, beta2 to 0.999, and epslion to 1e-8.

[0112] In a specific embodiment, the image classification using a pre-trained bilinear convolutional neural network model includes:

[0113] The preprocessed MRI images are divided into a first MRI image and a second MRI image;

[0114] A first network extracts a first feature from a first MRI image; a second network extracts a second feature from a second MRI image;

[0115] The first and second features are fused by dot product to obtain the final MRI image features;

[0116] The final MRI image features are classified using a classifier to obtain the final MRI image classification result.

[0117] Specifically, the first feature extracted by the first network from the first MRI image includes:

[0118] Feature extraction was performed on the first MRI image using convolutional layers;

[0119] The extracted first MRI image features are reduced in dimensionality using a max pooling layer;

[0120] For the first MRI image after dimensionality reduction, multi-scale features of the first MRI image are obtained by using residual block combination units;

[0121] By using a channel attention mechanism to add weights to the multi-scale features of the first MRI image, multi-scale features of the first MRI image with attention weights are obtained.

[0122] The first feature is obtained by using a pooling layer to reduce the dimensionality of the multi-scale features of the first MRI image with attention weights.

[0123] Specifically, the second network extracts a second feature from the second MRI image, including:

[0124] Feature extraction of the second MRI image was performed using convolutional layers;

[0125] The extracted second MRI image features are reduced in dimensionality using a max pooling layer.

[0126] For the dimension-reduced second MRI image, multi-scale features of the second MRI image are obtained by using residual block combination units;

[0127] By using a channel attention mechanism to add weights to the multi-scale features of the second MRI image, multi-scale features of the second MRI image with attention weights are obtained.

[0128] The second feature is obtained by using a pooling layer to reduce the dimensionality of the multi-scale features of the second MRI image with attention weights.

[0129] For the dimension-reduced first MRI image, multi-scale features of the first MRI image are obtained using residual block combination units, including:

[0130] The feature map of the first MRI image after 1x1 division is divided into s blocks according to the number of channels, and each block is x in size. i ;

[0131] The first convolution operation begins with the first block x1;

[0132] As i increases, the i-th block is added to the (i-1)-th block after convolution, and then the i-th convolution operation is performed.

[0133] By concatenating each convolutional segment, the multi-scale features of the first MRI image are obtained.

[0134] For the dimension-reduced second MRI image, multi-scale features of the second MRI image are obtained using residual block combination units, including:

[0135] For the dimension-reduced second MRI image, multi-scale features of the second MRI image are obtained using residual block combination units, including:

[0136] The feature map of the second MRI image after 1x1 is divided into s blocks according to the number of channels, and each block is x in size. i ;

[0137] The first convolution operation begins with the first block x1;

[0138] As i increases, the i-th block is added to the (i-1)-th block after convolution, and then the i-th convolution operation is performed.

[0139] By concatenating each convolutional segment, the multi-scale features of the second MRI image are obtained.

[0140] We selected the semi-public ADNI dataset, which contains over 200 patients and related images, and compared our model with commonly used medical image classification methods. The results show that our model has a significant advantage in diagnostic classification. By increasing the cardinality to extract multi-scale features, adding small residual blocks to the residual unit structure to enhance the connection between local and global blocks, and adding a CABW structure before aggregation after extraction, we achieved fine-grained information allocation and processing. By using an SVM classifier, we addressed the problem of limited medical image data samples and weak generalization ability. We conclude that our model is suitable for binary classification experiments in AD, NC, and MCI, achieving better classification accuracy and robustness.

[0141] Example 2

[0142] This embodiment provides an MRI image classification system based on feature extraction and attention mechanisms, including:

[0143] The image processing module is configured to acquire MRI image data and perform preprocessing.

[0144] The image classification module is configured to perform image classification using a pre-trained bilinear convolutional neural network model.

[0145] The bilinear convolutional neural network model includes a first network model, a second network model, and a classifier; the first network model and the second network model have the same structure, specifically including:

[0146] Convolutional layer, first max pooling layer, residual block combination unit and second max pooling layer;

[0147] The residual block combination unit is a multi-branch convolutional residual block;

[0148] An attention mechanism is added between the residual block combination unit and the second max pooling layer to add channel attention weights to the MRI image features processed by the residual block combination unit.

[0149] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0150] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0151] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0152] Example 3

[0153] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the MRI image classification method based on feature extraction and attention mechanisms as described in Embodiment 1 above.

[0154] Example 4

[0155] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the MRI image classification method based on feature extraction and attention mechanism as described in Embodiment 1 above.

[0156] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0157] This invention is described with reference to flowchart illustrations and block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and block diagrams, and combinations of blocks in the flowchart illustrations and block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and block diagrams. Figure 1 One or more processes and W or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and W or a box Figure 1 The function specified in one or more boxes.

[0159] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and W or a box Figure 1 The steps of the function specified in one or more boxes.

[0160] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0161] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for MRI image classification based on feature extraction and attention mechanism, characterized in that, The application relates to a method for classifying MRI images, comprising the following steps: acquiring MRI image data and performing preprocessing; using a pre-trained bilinear convolutional neural network model to perform image classification; wherein the bilinear convolutional neural network model comprises a first network model, a second network model and a classifier; the first network model and the second network model are the same in structure and specifically comprise: a convolutional layer, a maximum pooling layer, a residual block combination unit and a pooling layer; The residual block combination unit is a multi-branch convolution residual block; wherein the residual block combination unit adopts an improved ResNet model, changes the filter from a single branch to a multi-branch, uses the concept of 'base number' added in the improved 3x3 residual block to obtain the structure of the residual block combination unit; the image features obtained after preprocessing have a size of 224x224, and after a 7x7 convolution layer and a 3x3 max pooling, a size of 256x224x224 is obtained, the original residual block is first split into 32 groups according to the base number of the total channel number, and the 256x224x224 is divided into 32 groups through a 4x1 convolution kernel to obtain a feature of 4x224x224 size, then a 3x3 convolution kernel size is used to obtain a feature with a channel of 4, which plays a role in down-sampling the image data after conversion to achieve the effect of dimension reduction, and the features after dimension reduction are convolved to obtain a feature of 256x224x244, finally the aggregation effect is realized; again, the residual block with added base number is reduced and convolved, and the small residual block after splitting is replaced in the existing single structure of the residual, which is an improvement of the 3x3 residual block, and the fine-grained multi-scale parallelism is performed, the feature map after 1x1 is first divided into s blocks according to the channel number, s=4 is selected, each small block is x i size, the divided x i all perform a 3x3 convolution, and each 3x3 convolution operator receives feature information from the feature x i , assuming K( ) represents the convolution process, x1 is directly convolved at the beginning, with the increase of i, x i is added to K i-1 ( ) and then sent to K i ( ), and the cycle is performed in turn, and finally the split output block is concatenated; the attention mechanism is added between the residual block combination unit and the pooling layer, and the channel attention weight is added to the MRI image features processed by the residual block combination unit.

2. The MRI image classification method based on feature extraction and attention mechanism as claimed in claim 1, wherein, the step of acquiring MRI image data and performing preprocessing comprises: acquiring MRI image data; obtaining a two-dimensional MRI planar image based on the MRI image data; segmenting the two-dimensional MRI planar image to obtain a gray matter image, a white matter image and a cerebrospinal fluid image; performing spatial standardization and smoothing processing on the gray matter image, the white matter image and the cerebrospinal fluid image to obtain a preprocessed MRI image.

3. The MRI image classification method based on feature extraction and attention mechanism as claimed in claim 1, wherein, the step of using a pre-trained bilinear convolutional neural network model to perform image classification comprises: dividing the preprocessed MRI image into a first MRI image and a second MRI image; the first network extracts first features of the first MRI image; the second network extracts second features of the second MRI image; performing dot product fusion on the first features and the second features to obtain final MRI image features; using the classifier to classify the final MRI image features to obtain a final MRI image classification result.

4. The MRI image classification method based on feature extraction and attention mechanism as claimed in claim 3, wherein, the step of using the first network to extract first features of the first MRI image specifically comprises: using the convolutional layer to extract features of the first MRI image; using the maximum pooling layer to reduce the dimensionality of the extracted first MRI image features; for the reduced first MRI image, using the residual block combination unit to obtain multi-scale features of the first MRI image; using a channel attention mechanism to add weights to the multi-scale features of the first MRI image to obtain multi-scale features of the first MRI image with attention weights; using the pooling layer to reduce the dimensionality of the multi-scale features of the first MRI image with attention weights to obtain the first features.

5. The method for MRI image classification based on feature extraction and attention mechanism as claimed in claim 3, wherein, the step of using the second network to extract second features of the second MRI image specifically comprises: using the convolutional layer to extract features of the second MRI image; using the maximum pooling layer to reduce the dimensionality of the extracted second MRI image features; for the reduced second MRI image, using the residual block combination unit to obtain multi-scale features of the second MRI image; using a channel attention mechanism to add weights to the multi-scale features of the second MRI image to obtain multi-scale features of the second MRI image with attention weights; using the pooling layer to reduce the dimensionality of the multi-scale features of the second MRI image with attention weights to obtain the second features.

6. The method for MRI image classification based on feature extraction and attention mechanism as claimed in claim 4, wherein, for the reduced first MRI image, using the residual block combination unit to obtain multi-scale features of the first MRI image comprises: The feature maps of the first MRI image after 1x1 are averaged by channel number into s chunks, and each chunk size is ; From the first block Start the first convolution operation; along with i With the increase of , the i-th block and the convolution of the i-th block i Add the first -1 blocks and then proceed to the next step. i Convolutional operation; concatenating each convolutional segmentation block to obtain the multi-scale features of the first MRI image.

7. The MRI image classification method based on feature extraction and attention mechanism as claimed in claim 5, wherein, for the reduced second MRI image, using the residual block combination unit to obtain multi-scale features of the second MRI image comprises: for the reduced second MRI image, using the residual block combination unit to obtain multi-scale features of the second MRI image comprises: The feature maps of the second MRI image after 1x1 are divided into s blocks, and each block has a size of ; From the first block Start the first convolution operation; along with i With the increase of , the i-th block and the convolution of the i-th block i Add the first -1 blocks and then proceed to the next step. i Convolutional operation; After concatenating each convolutional segmented block, a multi-scale feature of the second MRI image is obtained.

8. A system for MRI image classification based on feature extraction and attention mechanism, characterized in that, The method comprises the steps of: an image processing module configured to acquire MRI image data and perform preprocessing; an image classification module configured to perform image classification using a pre-trained bilinear convolutional neural network model; The bilinear convolutional neural network model comprises a first network model, a second network model, and a classifier; the first network model and the second network model have the same structure and specifically comprise: a convolutional layer, a first max-pooling layer, a residual block combination unit, and a second max-pooling layer; The residual block combination unit is a multi-branch convolution residual block; wherein the residual block combination unit adopts an improved ResNet model, changes the filter from a single branch to a multi-branch, uses the concept of 'base number' added in the improved 3x3 residual block to obtain the structure of the residual block combination unit; the image features obtained after pre-processing have a size of 224x224, and after a 7x7 convolution layer and a 3x3 maximum pooling, a size of 256x224x224 is obtained, the original residual block is first split according to the whole channel number with a base number of 32, 256x224x224 is divided into 32 groups through a 4x1 convolution kernel to obtain a feature with a size of 4x224x224, then a 3x3 convolution kernel is used to obtain a feature with a channel of 4, the conversion plays a role in down-sampling of image data to realize dimension reduction effect, the features after dimension reduction are convolved to obtain a feature with a size of 256x224x244, and finally the aggregation effect is realized; again, the dimension reduction convolution in the residual block with the added base number is replaced by the small residual block after splitting in the existing single structure in the residual, which is an improvement of the 3x3 residual block, and the fine-grained multi-scale parallelism is performed, the feature map after 1x1 is first divided into s blocks according to the channel number, s=4 is selected, each small block is x i size, the divided x i all perform a 3x3 convolution, and each 3x3 convolution operator receives feature information from the feature x i , assuming that K( ) represents the convolution process, x1 is directly convolved at the beginning, with the increase of i, x i is added to K i-1 ( ) and then sent to K i ( ), the cycle is performed in turn, and finally the split output block is concatenated. An attention mechanism is added between the residual block combination unit and the second max-pooling layer to add weights to the MRI image features processed by the residual block combination unit.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the MRI image classification method based on feature extraction and attention mechanism in any one of claims 1-7.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the MRI image classification method based on feature extraction and attention mechanism in any one of claims 1-7.

Images