Medical image classification method based on MedConvMama model
By combining the MedConvMamba model with a state-space model and self-supervised learning, the problems of insufficient feature extraction and poor model performance in medical image classification are solved, achieving efficient and accurate medical image classification and improving the model's generalization ability and computational efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHEASTERN UNIV CHINA
- Filing Date
- 2025-12-26
- Publication Date
- 2026-04-21
AI Technical Summary
Existing medical image classification technologies suffer from insufficient feature extraction, poor model performance, and weak model generalization ability. In particular, they are computationally burdensome when processing large-size medical images in resource-constrained environments, making it difficult to process large amounts of image data quickly and efficiently.
The MedConvMamba model is adopted, which combines the Patch Embedding layer, SConvs-SSM Block, EmbeddingFusion module and fully connected layer. It is optimized by the self-supervised MM-MAE model, which takes advantage of the state space model and convolutional neural network to extract local fine-grained and global features, and reduces the dependence on labeled data through self-supervised learning strategy.
While maintaining linear computational complexity, it effectively extracts local fine-grained and global features, improving the model's performance and generalization ability, enhancing the accuracy and robustness of medical image classification, and reducing reliance on large amounts of labeled data.
Smart Images
Figure CN121904444A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of medical image classification, and more specifically, to a method for medical image classification based on the MedConvMamba model. Background Technology
[0002] Modern medical research relies heavily on medical imaging, which accurately reveals the structure and function of human tissues and organs, helping medical personnel explore patient conditions and serving clinical diagnosis and medical research. While various medical imaging methods are widely used with the development of medical technology, the accuracy of detecting and diagnosing diseases such as cancer remains significantly affected by the doctor's professional knowledge.
[0003] Deep learning has demonstrated tremendous potential in image processing, particularly in computer-aided inspection (CAD), becoming a mainstream technology. In medical image classification tasks, models based on convolutional neural networks (CNNs) and Transformers have performed exceptionally well and are widely used as backbone networks, encoders, or decoders. However, CNNs, due to limitations in their local receptive fields, struggle to capture long-range information in images, potentially leading to insufficient feature extraction. While Transformers perform well in modeling global information, their self-attention mechanism is computationally expensive, placing a heavy burden on processing large medical images and making them difficult to apply in resource-constrained environments, especially in clinical settings requiring rapid and efficient processing of large amounts of image data.
[0004] In recent years, state-space modeling (SSM) has attracted great interest from researchers. Modern SSMs, such as Mamba, can not only establish long-range dependencies, but also have computational complexity that is linear with the input size. SSM-based models have been extensively studied in various fields such as language understanding, general vision, and medical segmentation, but the performance of SSM-based medical image classification models still needs to be explored.
[0005] Therefore, we urgently need a novel medical image classification method to address the shortcomings of existing medical image classification technologies, such as insufficient feature extraction, poor model performance, and weak model generalization ability. Summary of the Invention
[0006] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.
[0007] Therefore, the purpose of this invention is to propose a method for medical image classification based on the MedConvMamba model.
[0008] To achieve the above objectives, the present invention provides a method for medical image classification based on the MedConvMamba model. The method includes: Step S1: Obtaining a medical image dataset; wherein the dimension of the medical image dataset is R. H×W×3 H and W are both positive integers; H is the height of each medical image in the medical image dataset; W is the width of each medical image in the medical image dataset; the medical image dataset is a labeled dataset; Step S2: Preprocess the medical image dataset; Step S3: Divide the preprocessed medical image dataset into a training set, a validation set, and a test set; Step S4: Build a medical image classification model; wherein, the medical image classification model is a MedConvMamba model; the MedConvMamba model includes: a Patch Embedding layer, L SConvs-SSM Blocks connected in sequence, an EmbeddingFusion module, and a first fully connected layer; wherein, each SConvs-SSM Block includes: a first convolutional layer and a Mamba Block; L is a positive integer; each Mamba Block includes: a second convolutional layer and a state space model; the Patch Embedding layer is used to divide the input medical image into 8×8 blocks. The non-overlapping patch is applied, and the dimensions of the medical image are mapped to C-dimensionality so that the dimensions of the medical image are R-dimensional. (H / 8)×(W / 8)×C Where C is a positive integer; the Patch Embedding layer is also used for processing elements of dimension R. (H / 8)×(W / 8)×C The medical image is subjected to layer normalization processing; the first convolutional layer is used to extract local features of the input medical image to output the output sequence F corresponding to the local features; the Mamba Block is used to receive the output sequence F and linearly project the output sequence F into components x and z respectively; where the dimension of components x and z is E; E is a positive integer; the Mamba Block is also used to perform a one-dimensional convolution operation on component x through the second convolutional layer to enhance the local dependency of the output sequence after the one-dimensional convolution operation; the Mamba Block is also used to perform a cubic linear projection on the output sequence after the one-dimensional convolution operation to obtain parameters B, C and Δ respectively; the Mamba Block is also used to calculate the hidden state h of the state space model based on the parameters B, C and Δ. tThe Mamba Block is further configured to combine the component z with the output sequence y of the state-space model using the SiLU activation function, ensuring that all non-overlapping patches output by each Mamba Block maintain the same shape as all non-overlapping patches output by the Patch Embedding layer, i.e., ensuring that the dimension of the sequence output by each Mamba Block is consistent with the output sequence F; the Embedding Fusion module is configured to receive the output of the last SConvs-SSM Block and perform processing on the last SConvs-SSM... The output of the Block performs a global average pooling operation to average the values of each feature channel across all spatial locations, thereby generating a global feature vector M with dimension 1×C; the first fully connected layer receives the feature vector M to automatically classify and predict medical images input to the MedConvMamba model; Step S5: Input the training set into the MedConvMamba model for training; Step S6: Input the validation set into the MedConvMamba model for optimization, and save the final optimized MedConvMamba model; Step S7: Input the test set into the final optimized MedConvMamba model to obtain the automatic classification result of the medical images corresponding to the test set.
[0009] Preferably, the method for medical image classification based on the MedConvMamba model is characterized by further comprising: Step S8: constructing a self-supervised MM-MAE model based on the MedConvMamba model; wherein the self-supervised MM-MAE model includes: an encoder and a decoder; the encoder is the MedConvMamba model; the decoder consists of a second fully connected layer and a third fully connected layer; the encoder is used to extract feature representations from the masked medical image; the decoder is used to reconstruct the original image data of the masked medical image from the feature representations; Step S9: employing a mask-based self-supervised pre-training strategy to train the self-supervised MM-MAE model. The model parameters are optimized; step S9 specifically includes: step S9.1: the encoder randomly masks the input unlabeled medical image dataset and extracts feature representations from the masked medical image dataset to output a feature vector G; step S9.2: the decoder reconstructs the original image data of the masked medical image dataset based on the feature vector G to predict the masked medical image dataset, and optimizes the parameters of the self-supervised MM-MAE model by minimizing the reconstruction loss function; step S10: the MedConvMamba model of the pre-trained self-supervised MM-MAE model is transferred to the downstream medical image classification task, and the MedConvMamba model is fine-tuned using a labeled medical image dataset to further optimize the MedConvMamba model, thereby achieving automatic classification and prediction of medical images from the further optimized MedConvMamba model.
[0010] Preferably, the kernel size in the first convolutional layer is 3, the stride is 1, and the number of channels is C.
[0011] Preferably, the value of the mapping dimension C is selected as either 192 or 384; the value of the dimension E is configured as 768.
[0012] Preferably, the expression corresponding to the reconstruction loss function is: (1) In equation (1), For rebuilding blocks; For real blocks; N The number of blocks that are covered.
[0013] The beneficial effects of this invention are: The method for medical image classification based on the MedConvMamba model provided in this invention has the following technical advantages: (1) Solved the problem of feature extraction: In view of the characteristics of medical images, namely, the overall structure is similar but the local lesion is highly variable, the existing CNN model and Transformer model are insufficient in feature extraction. Therefore, this invention develops a new medical image classification architecture, namely the MedConvMamba model, which can effectively extract local fine-grained features and global features while maintaining linear computational complexity. (2) Improved model performance: Explore the application potential of the CNN and SSM combined model in different modal medical images, optimize the parameter configuration and module settings of the model, improve the performance of the medical image classification model, and make it superior to the existing model in key indicators such as accuracy and sensitivity; (3) Enhanced model generalization ability: Given the difficulty in obtaining labeled medical image data, a self-supervised learning strategy is introduced to improve the generalization ability of the model by using unlabeled data, reducing the model's dependence on a large amount of labeled data, and making the model more robust in different medical image tasks.
[0014] Additional aspects and advantages of the invention will become apparent from the description which follows, or may be learned by practice of the invention. Attached Figure Description
[0015] Figure 1 A schematic flowchart of a method for medical image classification based on the MedConvMamba model according to an embodiment of the present invention is shown; Figure 2 A schematic diagram of the overall architecture of the MedConvMamba model according to an embodiment of the present invention is shown; Figure 3 A schematic diagram of the architecture of the Mamba Block in the MedConvMamba model according to an embodiment of the present invention is shown; Figure 4 A schematic diagram of the architecture of a self-supervised MM-MAE model according to an embodiment of the present invention is shown. Detailed Implementation
[0016] To better understand the above-mentioned objects, features, and advantages of the present invention, such as Figures 1 to 4 As shown in the accompanying drawings and specific embodiments, the present invention will be further described in detail below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0017] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.
[0018] Figure 1 A schematic flowchart illustrating a method for medical image classification based on the MedConvMamba model according to an embodiment of the present invention is shown. Figure 1 As shown, this method for medical image classification based on the MedConvMamba model includes: Step S1: Obtain a medical image dataset; wherein the dimension of the medical image dataset is R. H×W×3 H and W are both positive integers; H is the height of each medical image in the medical image dataset; W is the width of each medical image in the medical image dataset; the medical image datasets are all labeled datasets. Step S2: Preprocess the medical image dataset; Step S3: Divide the preprocessed medical image dataset into a training set, a validation set, and a test set; Step S4: Construct a medical image classification model; wherein, the medical image classification model is a MedConvMamba model; the MedConvMamba model includes: a Patch Embedding layer, L SConvs-SSM Blocks connected in sequence, an EmbeddingFusion module, and a first fully connected layer; wherein, each SConvs-SSM Block includes: a first convolutional layer and a Mamba Block; L is a positive integer; each Mamba Block includes: a second convolutional layer and a state space model; The Patch Embedding layer divides the input medical image into non-overlapping 8×8 patches and maps the dimensions of the medical image to C-dimensionality, so that the dimensions of the medical image are R. (H / 8)×(W / 8)×C Where C is a positive integer; the Patch Embedding layer is also used for layers of dimension R. (H / 8)×(W / 8)×C The medical image is subjected to layer normalization processing; the first convolutional layer is used to extract local features of the input medical image to output the output sequence F corresponding to the local features; the Mamba Block is used to receive the output sequence F and linearly project the output sequence F into components x and z respectively; where the dimension of components x and z is E; E is a positive integer; the Mamba Block is also used to perform a one-dimensional convolution operation on component x through the second convolutional layer to enhance the local dependency of the output sequence after the one-dimensional convolution operation; the Mamba Block is also used to perform a cubic linear projection on the output sequence after the one-dimensional convolution operation to obtain parameters B, C and Δ respectively; the Mamba Block is also used to calculate the hidden state h of the state space model based on the parameters B, C and Δ.t The output sequence y is combined with the component z by the SiLU activation function to ensure that all non-overlapping patches output by each Mamba Block have the same shape as all non-overlapping patches output by the PatchEmbedding layer, i.e., to ensure that the dimension of the sequence output by each Mamba Block is consistent with the output sequence F; the Embedding Fusion module receives the output of the last SConvs-SSMBlock and performs global average pooling on the output of the last SConvs-SSM Block to average the values of each feature channel at all spatial locations, thereby generating a global feature vector M with dimension 1×C; the first fully connected layer receives the feature vector M to achieve automatic classification and prediction of medical images input to the MedConvMamba model; Step S5: Input the training set into the MedConvMamba model for training; Step S6: Input the validation set into the MedConvMamba model for optimization, and save the final optimized MedConvMamba model; Step S7: Input the test set into the final optimized MedConvMamba model to obtain the automatic classification results of medical images corresponding to the test set.
[0019] In this embodiment, the method for medical image classification based on the MedConvMamba model provided by the present invention develops a novel medical image classification architecture, namely the MedConvMamba model, which can effectively extract local fine-grained features and global features while maintaining linear computational complexity, solving the problem of insufficient feature extraction in medical image classification models, and greatly improving the performance of medical image classification models.
[0020] In one embodiment of the present invention, the method for medical image classification based on the MedConvMamba model further includes: Step S8: Constructing a self-supervised MM-MAE model based on the MedConvMamba model; wherein the self-supervised MM-MAE model includes: an encoder and a decoder; the encoder is the MedConvMamba model; the decoder consists of a second fully connected layer and a third fully connected layer; the encoder is used to extract feature representations from the masked medical image; the decoder is used to reconstruct the original image data of the masked medical image from the feature representations; Step S9: Using a mask-based self-supervised pre-training strategy, the self-supervised MM-MAE is trained... The model parameters are optimized; step S9 specifically includes: step S9.1: the encoder randomly masks the input unlabeled medical image dataset and extracts feature representations from the masked medical image dataset to output a feature vector G; step S9.2: the decoder reconstructs the original image data of the masked medical image dataset based on the feature vector G to predict the masked medical image dataset, and optimizes the parameters of the self-supervised MM-MAE model by minimizing the reconstruction loss function; step S10: the MedConvMamba model of the pre-trained self-supervised MM-MAE model is transferred to the downstream medical image classification task, and the MedConvMamba model is fine-tuned using a labeled medical image dataset to further optimize the MedConvMamba model, thereby achieving automatic classification and prediction of medical images from the further optimized MedConvMamba model.
[0021] In one embodiment of the present invention, the kernel size of the first convolutional layer is 3, the stride is 1, and the number of channels is C.
[0022] In one embodiment of the present invention, the value of the mapping dimension C is selected as either 192 or 384; the value of the dimension E is configured as 768.
[0023] In one embodiment of the present invention, the expression corresponding to the reconstruction loss function is: (1) In equation (1), For rebuilding blocks; For real blocks; N The number of blocks that are covered.
[0024] The technical solution of the present invention will be illustrated below with a specific embodiment.
[0025] The specific embodiment of the method for medical image classification based on the MedConvMamba model is implemented through the following steps: (1) Step S1: Obtain the medical image dataset; wherein, the dimension of the medical image dataset is R. H×W×3 H and W are both positive integers; H is the height (i.e., vertical pixels) of each medical image in the medical image dataset; W is the width (i.e., horizontal pixels) of each medical image in the medical image dataset; the medical image datasets are all labeled datasets.
[0026] (2) Step S2: Preprocess the medical image dataset.
[0027] (3) Step S3: Divide the preprocessed medical image dataset into training set, validation set and test set.
[0028] (4) Step S4: Build a medical image classification model; wherein, the medical image classification model is the MedConvMamba model; such as Figure 2 and Figure 3 As shown, the MedConvMamba model includes: a Patch Embedding layer, L SConvs-SSM Blocks connected in sequence, an EmbeddingFusion module, and a Classifier; the Classifier is the first fully connected layer; each SConvs-SSM Block includes: a first convolutional layer and a Mamba Block; L is a positive integer; each Mamba Block includes: a second convolutional layer and a state space model; The Patch Embedding layer divides the input medical image into non-overlapping 8×8 patches and maps the dimensions of the medical image to C-dimensionality, so that the dimensions of the medical image are R. (H / 8)×(W / 8)×C C is a positive integer; the Patch Embedding layer is also used for layers of dimension R. (H / 8)×(W / 8)×CThe medical image is subjected to layer normalization processing; the first convolutional layer is used to extract local features of the input medical image to output the output sequence F corresponding to the local features; the kernel size of the first convolutional layer is 3, the stride is 1, and the number of channels is C; the value of C is 192 or 384; the Mamba Block is used to receive the output sequence F and linearly project the output sequence F into components x and z respectively; the dimension of components x and z is E; E is a positive integer; the value of E is 768; the Mamba Block is also used to perform a one-dimensional convolution operation on component x through the second convolutional layer to enhance the local dependency of the output sequence after the one-dimensional convolution operation; the Mamba Block is also used to perform a cubic linear projection on the output sequence after the one-dimensional convolution operation to obtain parameters B, C and Δ respectively; the Mamba Block is also used to calculate the hidden state h of the state space model based on parameters B, C and Δ. t The output sequence y is combined with the component z by the SiLU activation function to ensure that all non-overlapping patches output by each Mamba Block have the same shape as all non-overlapping patches output by the Patch Embedding layer, i.e., to ensure that the dimension of the sequence output by each Mamba Block is consistent with the output sequence F; the Embedding Fusion module receives the output of the last SConvs-SSM Block and performs global average pooling on the output of the last SConvs-SSM Block to average the values of each feature channel at all spatial locations, thereby generating a global feature vector M with dimension 1×C; the first fully connected layer receives the feature vector M to achieve automatic classification and prediction of medical images input to the MedConvMamba model; In step S4, the SConvs-SSM Block is the core module of the MedConvMamba model. The SConvs-SSMBlock design is concise and efficient, combining a single convolutional layer with the Mamba structure. The Mamba structure is a simplified SSM architecture, distinct from traditional linear attention modules and multilayer perceptron (MLP) module stacking methods. The Mamba structure integrates these two modules into a single module, simplifying the overall structure by replacing multiplication gating with activation functions and integrating the SSM transformation into the main path of the MLP. The Mamba design contains multiple such modules, equipped with standard normalized layers and spaced residual connections. This architecture not only maintains the linear scalability of the state-space model (SSM) in terms of sequence length but also possesses the modeling effect of a transformer. Mamba combines the advantages of CNNs and Transformers, providing an efficient foundational model for computer vision (CV). Unlike traditional Transformers, Mamba avoids the explicit storage of the entire context selection mechanism, and its one-dimensional and causal characteristics have become a research focus in computer vision applications. The pseudocode for the Mamba Block is shown in Table 1.
[0029] Table 1. Pseudocode of the Mamba Block As shown in Table 1, the structure of the Mamba Block's pseudocode clearly explains the functions of the SConvs-SSM Block and the Mamba Block mentioned in the above text. The pseudocode takes the input feature x as input and uses only two neural network parameters, A and D, for computation. This demonstrates the simplicity and efficiency of the module design, avoiding complex parameter settings and data preprocessing. The linear projection layer Linear(.) performs a cubic linear transformation on the input x to obtain parameters B, C, and Δ, reflecting the Mamba Block's integration of the functions of a traditional linear attention module and a multilayer perceptron (MLP) module, simplifying the overall structure through a unified linear projection operation.
[0030] In calculation At that time, exponential functions and matrix operations are used, and activation functions are used to replace multiplication gating, which further simplifies the structure. The recursive calculation process depends on the hidden state h from the previous time step. t-1 and the current input x t This reflects the characteristics of a state-space model, enabling the model to scale linearly in sequence length. The output y at each time step... tThe combination of these components into the final output y and its return demonstrates that the Mamba Block can model the entire sequence, exhibiting the modeling effect of a Transformer, capturing long-distance dependencies, and achieving efficient modeling. Furthermore, as shown in Table 1's pseudocode structure of the Mamba Block, it does not have an explicit mechanism for storing the entire context selection. It primarily processes sequences through recursive computation and linear transformations. Figure 3 As shown, each Mamba Block consists of a second convolutional layer and a state-space model (i.e., SSM). The Mamba Block combines the base blocks of the SSM with the MLP blocks popular in modern neural networks to form a new Mamba Block, which is stacked and combined with normalization and residual connections to form the Mamba Block.
[0031] In step S4, the SConvs-SSM Block is the core module of MedConvMamba. The SConvs-SSMBlock first uses a convolutional layer with a kernel size of 3, a stride of 1, and C channels to extract local features from the image, leveraging the advantages of CNNs in capturing local details. The output of the convolutional layer is then fed into the Mamba structure. In the Mamba structure, the normalized sequence is first linearly projected onto x and z (dimension E). x undergoes a one-dimensional convolution, followed by linear projection of the embedding to obtain B and C. A and B are then transformed using Δ, and finally, the output is calculated using SSM. z then selects and combines the results, ensuring the output maintains the same shape as the module input. This design combines the local feature extraction capabilities of CNNs with the large receptive field of the Mamba structure, demonstrating excellent performance in capturing subtle anomalies and contextual information in medical images, thus improving classification accuracy. The simplicity of the SConvs-SSMBlock not only simplifies the architecture but also reduces the number of parameters, making the model more lightweight and efficient.
[0032] (5) Step S5: Input the training set into the MedConvMamba model for training; In step S5, the choice of loss function during training has a profound impact on model performance. For the MedConvMamba model, the commonly used loss function is cross-entropy loss, especially in image classification tasks, and its formula is as follows: (2) In equation (2), The first is obtained by calculating the softmax function. iThe cross-entropy loss measures the difference between the predicted probability distribution of the model output and the actual label. A smaller value indicates that the model's prediction is closer to the true label. In multi-class classification problems, cross-entropy loss is optimized by calculating the log-likelihood difference between the predicted probability and the actual class label for each class. In this way, the loss function guides the model to adjust its parameters, thereby minimizing prediction error and improving classification accuracy. Cross-entropy loss is not only a commonly used standard choice in classification tasks but also effectively handles class imbalance problems, making the model more stable and robust when facing data from different classes.
[0033] (6) Step S6: Input the validation set into the MedConvMamba model for optimization, and save the final optimized MedConvMamba model.
[0034] (7) Step S7: Input the test set into the final optimized MedConvMamba model to obtain the automatic classification results of medical images corresponding to the test set.
[0035] In step S7, five key metrics—accuracy (ACC), precision, recall, specificity, and F1 score—were used to evaluate the classification performance of the MedConvMamba model. These metrics, widely recognized and used in medical image classification research, were calculated by representing true cases (TP), true negatives (TN), false negatives (FN), and false positives (FP) in the confusion matrix. The calculation of these evaluation metrics is as follows: (3) In Equation (3), Accuracy (ACC) demonstrates the correctness of the model in the overall classification task, reflects its comprehensive judgment ability on various types of samples, and allows researchers to understand the overall reliability of the model in medical image classification.
[0036] Precision reflects the accuracy with which the model predicts a positive class image, i.e., the proportion of images the model classifies as diseased that are actually diseased. It helps assess the model's accuracy in predicting positive samples and avoids overdiagnosis. Recall reflects the model's ability to identify all actual positive class images, i.e., the proportion of diseased images correctly identified. It helps determine if the model is missing many disease cases and is crucial for disease screening. Specificity demonstrates the model's ability to identify negative class images, i.e., the proportion of actually normal medical images correctly classified as normal. It can be used to assess the model's accuracy in classifying normal samples and prevent normal cases from being misclassified as diseased. The F1 score combines precision and recall, balancing the model's accuracy and comprehensiveness in classifying positive samples. It provides a more comprehensive evaluation of the model's performance in medical image classification tasks and helps select models that perform well across various aspects.
[0037] The hyperparameter settings for the MedConvMamba model method are as follows: Before starting network training, all images were resized to 224×224×3. Each image was normalized and standardized. We tuned the model parameters using the AdamW optimizer, setting the initial learning rate to 5e-4, weight decay to 1e-5, and employing cross-entropy loss. The model was trained for 300 iterations using the PyTorch framework with a batch size of 64. An early stopping strategy was implemented to prevent overfitting. No data augmentation techniques or pre-trained weights were used to ensure that the model's performance metrics directly reflect the unique architecture of the MedConvMamba model. The training setup included an Ubuntu 18.04 operating system and an NVIDIA RTX A6000 GPU. Comparative experiments showed that the hyperparameters of the research method were consistent with those of MedConvMamba.
[0038] Furthermore, this specific embodiment uses the publicly available COVID-19 medical imaging dataset for experiments and algorithm performance evaluation. The COVID-19 dataset is a chest X-ray image dataset. The COVID-19 dataset is a publicly available medical imaging database, primarily released by Qatar University, Dalhousie University, and ARK Information Solutions India. This database focuses on collecting chest X-ray images related to novel coronavirus infection to help researchers and doctors better understand and diagnose the virus through image analysis. To test the MedConvMamba model's ability as a baseline for medical image classification, comprehensive testing was conducted using different modalities of the COVID-19 dataset.
[0039] All images in the dataset were resized to 224×224×3. Each image underwent normalization and standardization, based on the mean and standard deviation of the ImageNet dataset. The input image data was standardized so that the pixel values of each channel followed a distribution with a mean of 0 and a standard deviation of 1. After standardization, the means for each channel (RGB) of the image were 0.485, 0.456, and 0.406, and the standard deviations were 0.229, 0.224, and 0.225, respectively. This improves the stability and convergence speed of model training. To ensure that the model performance metrics directly reflect the unique architecture of the MedConvMamba model, no data augmentation techniques were used.
[0040] To demonstrate the potential of the MedConvMamba model, it was compared with state-of-the-art Mamba-based medical image classification models and some other state-of-the-art models applied to medical image classification tasks. Experimental results are shown in Table 2. Table 2. Comparison of classification performance of different models in the COVID-19 database of radiographic images of novel coronavirus infection. The results demonstrate that despite its simple architecture, the MedConvMamba model performs exceptionally well in medical image classification tasks. The MedConvMamba model outperforms mainstream Mamba-based models, as well as CNN- and Transformer-based models, in terms of accuracy (ACC) on the selected dataset. Notably, its sensitivity and F1 score also surpass those of other models. Sensitivity and F1 score are crucial metrics in medical image classification tasks. This is because multi-class disease diagnosis models typically place greater emphasis on capturing diseased samples. Misdiagnosis in medical diagnosis (i.e., incorrectly classifying diseased samples as healthy samples) can have serious consequences, therefore, the model needs to minimize such errors. Clearly, the superior performance of the MedConvMamba model in both sensitivity and F1 score further demonstrates its advantages in medical image classification tasks.
[0041] (8) Step S8: Build a self-supervised MM-MAE model based on the MedConvMamba model; such as Figure 4 As shown, the self-supervised MM-MAE model includes an encoder and a decoder; the encoder is the MedConvMamba model; the decoder consists of a second fully connected layer and a third fully connected layer. The encoder is used to extract feature representations from the masked medical image; The decoder is used to reconstruct the original image data of the masked medical image from the feature representation.
[0042] In step S8, the Mask Self-Supervised Learning (MSSL) framework employed in the self-supervised MM-MAE model uses the MedConvMamba model as the encoder to extract robust features from medical images. It utilizes two simple fully connected second and third fully connected layers as the decoder, facilitating the reconstruction of the original image data from the encoded features. Unlike traditional supervised learning, which requires large labeled datasets, MSSL pre-trains the model using unlabeled data, enabling it to learn generalized features transferable to various downstream tasks. Specifically, the input medical image is first pre-processed into blocks, then the model randomly masks certain regions. The unmasked regions are then embedded into a high-dimensional feature space via linear mapping and input into the MedConvMamba encoder. This encoder incorporates a simplified state-space model. The SSM (Self-Structured Array) and convolutional layers capture local details and long-term dependencies to extract key features. Then, the decoder receives the encoder output and the embedding information of the masked regions, and reconstructs a near-original complete image through fusion operations.
[0043] (8) Step S9: Optimize the parameters of the self-supervised MM-MAE model by adopting a mask-based self-supervised pre-training strategy; Step S9 specifically includes: Step S9.1: Randomly masking the input unlabeled medical image dataset using the encoder, and extracting feature representations from the masked medical image dataset to output a feature vector G; Step S9.2: Reconstructing the original image data of the masked medical image dataset based on the feature vector G using the decoder to predict the masked medical image dataset, and optimizing the parameters of the self-supervised MM-MAE model by minimizing the reconstruction loss function; The expression corresponding to the reconstruction loss function is: (1) In equation (1), For rebuilding blocks; For real blocks; N The number of blocks that are covered.
[0044] In step S9, MedConvMamba integrates a simplified state-space model (SSM) with convolutional layers, combining the advantages of CNNs and transformers to capture local details and long-term dependencies crucial for tasks such as disease classification and lesion detection. During pre-training, a portion of the input image is randomly masked, and the model learns to predict the missing pixels based on the surrounding context, thereby facilitating the extraction of semantically meaningful features.
[0045] (9) Step S10: Transfer the MedConvMamba model of the pre-trained self-supervised MM-MAE model to the downstream medical image classification task, and fine-tune the MedConvMamba model with a labeled medical image dataset to further optimize the MedConvMamba model, thereby realizing the automatic classification and prediction of medical images of the MedConvMamba model after further optimization of the input.
[0046] In step S10, the core Mamba architecture of MedConvMamba replaces the traditional gating mechanism with an activation function and incorporates the SSM transformation into the MLP path, supported by normalization layers and residual connections. Therefore, after pre-training on unlabeled data, MedConvMamba is fine-tuned on task-specific labeled datasets. This architecture reduces reliance on large labeled datasets while improving performance across a range of medical imaging tasks.
[0047] To verify the performance of the self-supervised MM-MAE model built with MedConvMamba as the backbone, pre-training was performed on the COVID-19 dataset, and fine-tuning was conducted on the Chest X-ray Image dataset. The COVID-19 dataset is a publicly available medical imaging database, primarily released by Qatar University, Dalhousie University, and ARK Information Solutions in India. The Chest X-ray Image dataset contains chest X-ray images of patients who tested positive for COVID-19 or were suspected of having other viral and bacterial pneumonias (MERS, SARS, and ARDS). The images were obtained from a retrospective cohort of pediatric patients aged one to five years at Guangzhou Women and Children's Medical Center and from public datasets of typical chest X-ray examinations from the NIH, Stanford University, MIT, and Indiana University. The COVID-19 dataset was used for pre-training of the self-supervised MM-MAE model, and the Chest X-ray Image dataset was used as an external dataset for fine-tuning experiments. The hyperparameter settings for the pre-training stage of the MM-MAE method and its comparative experimental methods are shown in Table 3, and the hyperparameter settings for the fine-tuning stage of the MM-MAE method and its comparative experimental methods are shown in Table 4.
[0048] Table 3. Hyperparameter settings during the pre-training phase of the MM-MAE method and its comparative experiments. Table 4. Hyperparameter settings during the fine-tuning stage of the MM-MAE method and its comparative experimental methods. Before starting network training, all images except the MAGE model were resized to 224×224×3. The MAGE model, due to its use of VQGAN weights for image representation, required image resizing to 256×256×3 to meet the publicly disclosed VQGAN weight requirements. Each image underwent normalization and standardization. All models used the PyTorch framework, and an early stopping strategy was implemented to prevent overfitting. No data augmentation techniques or pre-trained weights were used. The training setup included an Ubuntu 18.04 operating system and an NVIDIA RTX A6000 GPU.
[0049] This specific implementation compares the general representation capabilities of supervised learning, MAE, MAGE, and ConvNextV2 for medical image vision through cross-dataset, cross-disease transfer learning tasks. Specifically, the aforementioned masked self-supervised model is pre-trained on the COVID-19 dataset to obtain corresponding pre-trained parameters. Then, the obtained pre-trained weights are used to fine-tune further training on the Chest X-ray Image dataset to adapt the model to the specific task. The performance is then compared with that of the MedConvMamba model, which is directly trained on the Chest X-ray Image dataset without using a self-supervised architecture. The comparison results are shown in Table 5.
[0050] Table 5. Impact of different pre-training dataset mask rates on fine-tuning performance Table 5 shows the fine-tuning performance of each model on the Chest X-ray Image dataset. The evaluation results indicate that masked self-supervised learning significantly outperforms traditional supervised learning methods. MedConvMamba, as a supervised learning model, has the weakest performance among all models, with an accuracy of 94.95% and an F1 score of 94.97%. MAE and MAGE, as masked self-supervised models, show more stable performance. MAGE outperforms MAE in accuracy and F1 score (95.69% and 95.58%), demonstrating the advantage of improved masking strategies. ConvNextV2 performs well in accuracy (96.57%) and score (96.28%), but its accuracy (94.96%) is still lower than other masked self-supervised models. The masked self-supervised MM-MAE model, based on the MedConvMamba skeleton, performs best across all metrics, with accuracy, precision, sensitivity, and F1 score of 96.43%, 96.46%, 96.43%, and 96.44%, respectively, and a specificity of 100%. This result demonstrates that masked self-supervised learning significantly improves the model's representational ability and performance in downstream tasks by effectively combining medical image feature optimization and transfer learning, showcasing the advantages of the SSM model in masked self-supervised learning.
[0051] In summary, the method for medical image classification based on the MedConvMamba model provided by this invention has unparalleled technical advantages.
[0052] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for medical image classification based on the MedConvMamba model, characterized in that, include: Step S1: Obtain a medical image dataset; wherein the dimension of the medical image dataset is R. H×W×3 H and W are both positive integers; H is the height of each medical image in the medical image dataset; W is the width of each medical image in the medical image dataset; the medical image datasets are all labeled datasets. Step S2: Preprocess the medical image dataset; Step S3: Divide the preprocessed medical image dataset into a training set, a validation set, and a test set; Step S4: Construct a medical image classification model; wherein the medical image classification model is a MedConvMamba model; the MedConvMamba model includes: a Patch Embedding layer, L SConvs-SSM Blocks connected in sequence, an Embedding Fusion module, and a first fully connected layer; wherein each SConvs-SSM Block includes: a first convolutional layer and a Mamba Block; L is a positive integer; each Mamba Block includes: a second convolutional layer and a state space model; The Patch Embedding layer is used to divide the input medical image into non-overlapping patches of size 8×8, and map the dimensions of the medical image to C-dimensionality, so that the dimensions of the medical image are R. (H / 8)×(W / 8)×C Where C is a positive integer; The Patch Embedding layer is also used for processing elements of dimension R. (H / 8)×(W / 8)×C The medical images were subjected to layer normalization processing; The first convolutional layer is used to extract local features of the input medical image to output the output sequence F corresponding to the local features; The Mamba Block is used to receive the output sequence F and linearly project the output sequence F into components x and z; wherein the dimension of components x and z is E; and E is a positive integer. The Mamba Block is also used to perform a one-dimensional convolution operation on component x through the second convolutional layer to enhance the local dependencies of the output sequence after the one-dimensional convolution operation; The Mamba Block is also used to perform a three-dimensional linear projection on the output sequence after the one-dimensional convolution operation to obtain parameters B, C and Δ respectively. The Mamba Block is also used to calculate the hidden state h of the state-space model based on the parameters B, C, and Δ. t and its output sequence y; The Mamba Block is also used to combine the component z with the output sequence y of the state space model through the SiLU activation function to ensure that all non-overlapping patches output by each Mamba Block have the same shape as all non-overlapping patches output by the Patch Embedding layer, that is, to ensure that the dimension of the sequence output by each Mamba Block is consistent with the output sequence F. The Embedding Fusion module is used to receive the output of the last SConvs-SSM Block and perform a global average pooling operation on the output of the last SConvs-SSM Block to average the values of each feature channel at all spatial locations, thereby generating a global feature vector M with a dimension of 1×C. The first fully connected layer is used to receive the feature vector M in order to automatically classify and predict medical images input to the MedConvMamba model; Step S5: Input the training set into the MedConvMamba model for training; Step S6: Input the validation set into the MedConvMamba model for optimization, and save the final optimized MedConvMamba model; Step S7: Input the test set into the final optimized MedConvMamba model to obtain the automatic medical image classification results corresponding to the test set.
2. The method for medical image classification based on the MedConvMamba model according to claim 1, characterized in that, Also includes: Step S8: Construct a self-supervised MM-MAE model based on the MedConvMamba model; wherein the self-supervised MM-MAE model includes: an encoder and a decoder; the encoder is the MedConvMamba model; the decoder consists of a second fully connected layer and a third fully connected layer; The encoder is used to extract feature representations from the masked medical image; The decoder is used to reconstruct the original image data of the masked medical image from the feature representation; Step S9: Optimize the parameters of the self-supervised MM-MAE model using a mask-based self-supervised pre-training strategy; Step S9 specifically includes: Step S9.1: Randomly mask the input unlabeled medical image dataset using the encoder, and extract feature representations from the masked medical image dataset to output a feature vector G; Step S9.2: Reconstruct the original image data of the masked medical image dataset based on the feature vector G using the decoder to predict the masked medical image dataset, and optimize the parameters of the self-supervised MM-MAE model by minimizing the reconstruction loss function; Step S10: Transfer the MedConvMamba model, which is a pre-trained self-supervised MM-MAE model, to the downstream medical image classification task, and fine-tune the MedConvMamba model with a labeled medical image dataset to further optimize the MedConvMamba model, thereby enabling automatic classification and prediction of medical images based on the further optimized MedConvMamba model.
3. The method for medical image classification based on the MedConvMamba model according to claim 1, characterized in that, The kernel size in the first convolutional layer is 3, the stride is 1, and the number of channels is C.
4. The method for medical image classification based on the MedConvMamba model according to claim 1, characterized in that, The value of the mapping dimension C is selected from either 192 or 384; the value of the dimension E is configured to be 768.
5. The method for medical image classification based on the MedConvMamba model according to any one of claims 1 to 4, characterized in that, The expression corresponding to the reconstruction loss function is: （1） In equation (1), For rebuilding blocks; For real blocks; N The number of blocks that are covered.