A medical image classification method and system of a structure-aware state space model

Abstract

The application discloses a kind of structural perception state space model medical image classification method and system, it is related to artificial intelligence and medical image analysis field.The system includes: input adaptation module, for uniform processing different modal medical image;Multi-scale feature fusion module, for through multi-inflation rate convolution branch extraction and fusion multi-scale lesion feature;Structural perception state space modeling module, for introducing edge perception convolution in state space model to modulate state update, enhance the modeling ability to anatomical structure and boundary;Dynamic channel gating module, for through gating mechanism adaptive enhancement key semantic channel, output semantic enhancement feature;Classification output module, for output final disease classification result.The application solves the problems of poor cross-task transfer performance, weak modal compatibility and insufficient structure modeling capability in the prior art, and achieves excellent performance in various medical image classification tasks such as chest X-ray and dermatoscope.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and medical image analysis, and more specifically to a medical image classification method and system based on a structure-aware state-space model. Background Technology

[0002] In recent years, artificial intelligence has been widely applied in the field of medical image analysis, especially deep learning-based automatic diagnostic technology, which has made significant progress in disease detection, segmentation, and classification. Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) are currently the mainstream frameworks for medical image classification. CNNs can extract spatial features through local convolutions, making them suitable for medical image scenarios with obvious structured features, while ViTs rely on self-attention mechanisms to achieve global feature modeling, demonstrating outstanding performance in tasks such as skin lesion classification and chest disease recognition. Meanwhile, State Space Models (SSMs) have been introduced into visual tasks, attracting attention for their linear time complexity and ability to model long-range dependencies. Typical examples include the Mamba series models (Gu & Dao, 2023) and the visual version, MambaVision (Hatamizadeh & Kautz, 2024). Although these models perform well in natural image recognition, medical imaging scenarios present unique challenges, such as modal differences (grayscale vs. color), variable lesion scale, blurred boundaries, and scarce annotations, making it difficult for general-purpose models to be directly adapted to medical tasks.

[0003] Therefore, how to generate a medical image classification method that can uniformly process medical images of different modalities, explicitly model anatomical structures, and has strong cross-task generalization ability is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a medical image classification method and system based on a structure-aware state-space model that overcomes or at least partially solves the above problems.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] This invention provides a medical image classification system, specifically comprising: an input adaptation module, a multi-scale feature fusion module, a structure-aware state space modeling module, a dynamic channel gating module, and a classification output module connected in sequence.

[0007] The input adaptation module receives medical image data of different modalities, performs unified feature projection processing on the medical image data through an adaptive convolutional layer, outputs a feature map, and passes the feature map to the multi-scale feature fusion module.

[0008] The multi-scale feature fusion module extracts and fuses lesion features of different scales through convolutional branches with different dilation rates, outputs fused features, and inputs the fused features output by the multi-scale feature fusion module into the structure-aware state space modeling module.

[0009] The structure-aware state-space modeling module extracts edge features from medical images through edge-aware convolutional layers, generates structural features, and inputs the structural features generated by the structure-aware state-space modeling module into the dynamic channel gating module.

[0010] The dynamic channel gating module performs semantic screening and weighting of diseases through a dynamic gating mechanism, outputs semantically enhanced features, and inputs the semantically enhanced features output by the dynamic channel gating module into the classification output module;

[0011] The classification output module processes semantically enhanced features and outputs the disease classification results for medical images.

[0012] Preferably, the medical image data received by the input adapter module includes single-channel grayscale image data and three-channel color image data.

[0013] Preferably, the multi-scale feature fusion module is used to enhance the perception of lesions at different scales without increasing computational complexity. Specifically, it includes:

[0014] Dilated convolution operations are performed on the input feature maps. The dilated convolution branches have different dilation rates. The multi-scale features output by the dilated convolution branches are then weighted and fused using learnable weights to output fused features. The relevant calculation formula is as follows:

[0015]

[0016] in, It is the input feature map, and Conv(·) represents the dilated convolution operation. The void ratio, Here, is the learnable fusion coefficient, and h is the input feature tensor.

[0017] Preferably, the structure-aware state-space modeling module extracts edge features from medical images through edge-aware convolution to generate structural features, specifically including:

[0018] Edge and structural features are extracted from medical images using edge-aware convolutional layers and the SiLU activation function. These features are then input into the update equation of the structure-aware state-space modeling module. The update equation is as follows:

[0019]

[0020]

[0021] For structural features, Let A be the edge feature, B be the state transition moment, C be a convolution operation, ΔA be the change in the state transition matrix A, and I be the identity matrix.

[0022] Preferably, the dynamic channel gating module performs semantic filtering and weighting through a dynamic gating mechanism, and outputs semantically enhanced features, specifically including:

[0023] The dynamic channel gating module adaptively adjusts the channel weights based on features, enhances disease-related semantic features, suppresses background noise, and outputs semantically enhanced features. Its calculation method is as follows:

[0024]

[0025] Where x is the input vector, Weight matrix, and Responsible for dimensional increase / transformation, It is responsible for projecting the dimensions back to their original size. (·) is the activation function, ⊙ represents element-wise multiplication, σ is the Sigmoid function, and y is the output vector.

[0026] Preferably, the classification output module processes the semantic enhancement features through a global average pooling layer and a fully connected layer to output the disease classification results of medical images.

[0027] This invention provides a medical image classification method based on a structure-aware state-space model according to the above system, specifically including:

[0028] S1. Preprocess the input medical image data;

[0029] S2. Input the preprocessed medical image data into the input adaptation module for feature projection processing, and then pass the processed feature map to the multi-scale feature fusion module for multi-scale feature extraction and fusion.

[0030] S3. Input the fused multi-scale features into the structure-aware state space modeling module. Extract edge features from medical images through edge-aware convolution to generate an edge response map. Combine the edge response map with the edge features and structural features, and use the edge response map as a weighting factor to perform weighted processing on the edge features and structural features, thereby incorporating structural information into the state update process and generating structural features.

[0031] S4. Adaptively adjust the channel weights based on the features to enhance disease-related semantic features, suppress background noise, and output semantically enhanced features;

[0032] S5. After processing by a global average pooling layer and a fully connected layer, the medical image disease classification results are output.

[0033] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a medical image classification method and system based on a structure-aware state-space model. The beneficial effects of the above technical solutions provided by the present invention include at least the following:

[0034] In terms of modal compatibility, an adaptive input projection mechanism is adopted to achieve unified modeling of grayscale and color images; in terms of structural modeling, a structure-aware state update mechanism is used to improve the structural sensitivity of the model; in terms of multi-scale feature extraction, multi-dilated convolution branches are used to achieve efficient fusion of multi-scale information; in terms of task transfer capability, the unified architecture enables transfer applications between different medical imaging tasks; in terms of performance and efficiency, it demonstrates high accuracy, efficiency and generalization. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0036] Figure 1 This is a schematic diagram of the overall structure of the medical image classification system provided in this embodiment of the invention;

[0037] Figure 2 This is a comparison chart of the average AUROC performance of different models provided in the embodiments of the present invention on the CheXpert chest X-ray dataset. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] This invention discloses a medical image classification method and system based on a structure-aware state-space model, specifically including the following:

[0040] This invention provides a medical image classification system, specifically comprising: an input adaptation module, a multi-scale feature fusion module, a structure-aware state space modeling module, a dynamic channel gating module, and a classification output module connected in sequence.

[0041] The input adaptation module receives medical image data of different modalities, performs unified feature projection processing on the medical image data through an adaptive convolutional layer, outputs a feature map, and passes the feature map to the multi-scale feature fusion module.

[0042] The multi-scale feature fusion module extracts and fuses lesion features of different scales through convolutional branches with different dilation rates, outputs fused features, and inputs the fused features output by the multi-scale feature fusion module into the structure-aware state space modeling module.

[0043] The structure-aware state-space modeling module extracts edge features from medical images through edge-aware convolutional layers, generates structural features, and inputs the structural features generated by the structure-aware state-space modeling module into the dynamic channel gating module.

[0044] The dynamic channel gating module performs semantic screening and weighting of diseases through a dynamic gating mechanism, outputs semantically enhanced features, and inputs the semantically enhanced features output by the dynamic channel gating module into the classification output module;

[0045] The classification output module processes semantically enhanced features and outputs the disease classification results for medical images.

[0046] In one specific embodiment, the overall structure and implementation process of the system are as follows:

[0047] like Figure 1 As shown, the medical image classification system based on a structure-aware state-space model provided by this invention is implemented through the following steps:

[0048] S1: Input medical images and perform input adaptation processing. Input the medical images to be classified into the system. The medical images can be single-channel grayscale medical images or three-channel color medical images.

[0049] The input adaptation module performs unified feature mapping on medical images with different numbers of channels, so that the output is a feature representation with a unified number of channels and feature dimensions.

[0050] S2: Feature extraction and hierarchical state space modeling. The features processed by the input adaptation module are sequentially input into multiple cascaded state space modeling layers (MixMamba layers 1 to 4). Each layer includes:

[0051] 1. Convolutional module, used to extract local spatial features;

[0052] 2. State-space modeling module, used for long-range dependency modeling of feature sequences;

[0053] 3. Structure-aware state space module, used to incorporate prior structural information from medical images during state updates.

[0054] Among them, the feature map spatial resolution is reduced step by step through the downsampling module between different layers to improve the semantic expression capability.

[0055] S3: Multi-scale feature fusion. In the state space modeling process, a multi-scale feature fusion module is introduced. The feature maps are processed in parallel using dilated convolutions with different dilation rates, and then weighted and fused using learnable weights, thereby enhancing the model's ability to express lesion features at different scales.

[0056] The multi-scale feature fusion module is used to enhance the perception of lesions at different scales without increasing computational complexity. Specific components include:

[0057] Dilated convolution operations are performed on the input feature maps. The dilated convolution branches have different dilation rates. The multi-scale features output by the dilated convolution branches are then weighted and fused using learnable weights to output fused features. The relevant calculation formula is as follows:

[0058]

[0059] in, It is the input feature map, and Conv(·) represents the dilated convolution operation. The void ratio, Here, is the learnable fusion coefficient, and h is the input feature tensor.

[0060] S4: Structure-aware state-space modeling. The structure-aware state-space modeling module extracts edge features from medical images through edge-aware convolution to generate structural features, specifically including:

[0061] Edge and structural features are extracted from medical images using edge-aware convolutional layers and the SiLU activation function. These features are then input into the update equation of the structure-aware state-space modeling module. The update equation is as follows:

[0062]

[0063]

[0064] For structural features, Let A be the edge feature, B be the state transition moment, C be a convolution operation, ΔA be the change in the state transition matrix A, and I be the identity matrix.

[0065] The features output from the last-layer state space modeling module are input into the global feature aggregation module, and the spatial dimension is compressed through pooling operation to obtain the global feature vector.

[0066] S5: Dynamic channel gating and classification output. The dynamic channel gating module performs semantic filtering and weighting through a dynamic gating mechanism, outputting semantically enhanced features, specifically including:

[0067] The dynamic channel gating module adaptively adjusts the channel weights based on features, enhances disease-related semantic features, suppresses background noise, and outputs semantically enhanced features. Its calculation method is as follows:

[0068]

[0069] Where x is the input vector, Weight matrix, and Responsible for dimensional increase / transformation, It is responsible for projecting the dimensions back to their original size. (·) is the activation function, ⊙ represents element-wise multiplication, σ is the Sigmoid function, and y is the output vector.

[0070] The global feature vector is input into the dynamic channel gating module, and the feature channels are dynamically weighted by the gated multilayer perceptron to highlight the semantic features related to pathology and suppress redundant channels. Finally, the classification layer outputs the classification result of the medical image.

[0071] Specifically, the modulation of the state update process involves the edge-aware convolutional module performing convolution operations on the edge features of the input image during each state update. This generates an edge response map. Then, by combining these edge response maps with edge and structural features, they act as weighting factors to modulate the state update process, causing the model to focus more on anatomical structures and lesion boundaries in the image during learning. This modulation method enables the model to better understand structural information in medical images, especially when lesion boundaries are blurred or anatomical structures are complex, thus improving classification and localization accuracy.

[0072] The specific steps for updating the state are as follows:

[0073] 1. Edge-aware convolution: Extracts edge features from an image and generates an edge response map.

[0074] 2. State Update Modulation: The process of modulating state transitions by weighting edge features and structural features using the generated edge response map.

[0075] 3. Enhance structural sensitivity: By adjusting the state update process, the model's ability to perceive anatomical structures and lesion boundaries is enhanced, thereby improving the accuracy of lesion localization.

[0076] The dynamic channel gating module is used for channel-level feature filtering and weighting. It highlights lesion-related semantic features and suppresses background noise through a dynamic gating mechanism.

[0077] The classification output module contains a global average pooling layer and a fully connected layer to generate single-label or multi-label classification results.

[0078] The present invention also provides another embodiment:

[0079] In this embodiment, to verify the cross-task classification performance of the present invention, chest X-ray data from the publicly available dataset CheXpert-v1.0-small and dermoscopy image data from HAM10000 were selected for a comparative experiment.

[0080] The experiments used the same training hyperparameters, such as a learning rate of 1e-4, a batch size of 256, and the AdamW optimizer, and were conducted on the same hardware platform.

[0081] First, this experiment uses a comparative setup, selecting current mainstream models as the comparison method:

[0082] EfficientNet-B0: An efficient convolutional neural network, a convolutional neural network (CNN) optimized through compound scaling.

[0083] CheXNet (DenseNet121): A convolutional medical image classification model;

[0084] Swin Transformer: A visual model based on hierarchical self-attention;

[0085] ViT-B / 16: Standard Vision Transformer;

[0086] MambaVision-T: A visual model based on state-space modeling;

[0087] Swin-T: A window-moving visual Transformer, a visual model based on hierarchical self-attention, and a lightweight version of the Swin Transformer series.

[0088] The MixMamba model proposed in this invention is an innovative model that combines the Mamba (State Space Model, SSM) architecture with modern visual processing technology. Through its efficient long sequence modeling capabilities, it can accurately capture complex global anatomical features in medical images while maintaining low computational complexity.

[0089] Then, this experiment compares the performance results, such as... Figure 2 (a) and Figure 2 As shown in (b), the bar chart illustrates the average AUROC performance of each model across 14 disease categories in the CheXpert chest X-ray dataset. The figure demonstrates that the proposed MixMamba model achieves the highest AUROC across most disease categories, with an average performance improvement of approximately 3.5% compared to traditional CNN models and approximately 2.1% compared to Transformer models. This indicates that the proposed model possesses superior structure awareness and generalization performance in cross-task medical image classification.

[0090] Finally, the performance results were analyzed in this experiment. The results show that the structure-aware state-space modeling method proposed in this invention can simultaneously model the local structure and global dependencies of lesions; the multi-scale feature extraction module enhances the fusion capability of multi-scale lesion features, and the structure-aware state-space modeling module improves the accuracy of boundary and morphological information recognition; the dynamic channel gating module further improves the semantic expression capability of the model through channel weighting. Therefore, the scheme of this invention can maintain stable and superior performance in medical image classification tasks of different tasks and different modalities.

[0091] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0092] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A medical image classification system based on a structure-aware state-space model, characterized in that, The medical image classification system specifically includes: an input adaptation module, a multi-scale feature fusion module, a structure-aware state space modeling module, a dynamic channel gating module, and a classification output module connected in sequence; The input adaptation module is used to receive medical image data of different modalities, perform unified feature projection processing on the medical image data through an adaptive convolutional layer, output a feature map, and pass the feature map to the multi-scale feature fusion module. The multi-scale feature fusion module extracts and fuses lesion features of different scales from the feature map through dilated convolution branches with different dilation rates, outputs fused features, and inputs the fused features output by the multi-scale feature fusion module into the structure-aware state space modeling module. The structure-aware state-space modeling module extracts edge and structural features from medical images through edge-aware convolutional layers and SiLU activation functions, and inputs the structural features generated by the structure-aware state-space modeling module into the dynamic channel gating module. The dynamic channel gating module performs semantic screening and weighting of diseases through a dynamic gating mechanism, outputs semantically enhanced features, and inputs the semantically enhanced features output by the dynamic channel gating module into the classification output module; The classification output module processes the semantic enhancement features and outputs the disease classification results of the medical image. The structure-aware state-space modeling module extracts edge features from medical images through edge-aware convolution to generate structural features, specifically including: Edge and structural features are extracted from medical images using edge-aware convolutional layers and the SiLU activation function. These edge and structural features are then input into the update equation of the structure-aware state-space modeling module. The update equation is as follows: For structural features, For edge features, A is the state transition moment, B is the weight matrix, C is a convolution operation, ΔA is the change in the state transition matrix A, and I is the identity matrix; The dynamic channel gating module performs semantic filtering and weighting through a dynamic gating mechanism, and outputs semantically enhanced features, specifically including: The dynamic channel gating module adaptively adjusts the channel weights based on features, enhances disease-related semantic features, suppresses background noise, and outputs semantically enhanced features. Its calculation method is as follows: Where x is the input vector, Weight matrix, and Responsible for dimensional increase / transformation, Responsible for projecting dimensions back to their original size. (·) is the activation function, ⊙ represents element-wise multiplication, σ is the Sigmoid function, and y is the output vector.

2. The medical image classification system based on a structure-aware state-space model according to claim 1, characterized in that, The medical image data processed by the input adaptation module includes single-channel grayscale image data and three-channel color image data.

3. The medical image classification system based on a structure-aware state-space model according to claim 1, characterized in that, The multi-scale feature fusion module is used to enhance the perception of lesions at different scales without increasing computational complexity. Specifically, it includes: A dilated convolution operation is performed on the input feature map. The dilated convolution branches have different dilation rates. The multi-scale features output by the dilated convolution branches are then weighted and fused with learnable weights to output fused features. The relevant calculation formula is as follows: in, It is the input feature map, and Conv(·) represents the dilated convolution operation. The void ratio, Here, is the learnable fusion coefficient, and h is the input feature tensor.

4. The medical image classification system based on a structure-aware state-space model according to claim 1, characterized in that, The classification output module processes the semantic enhancement features, specifically including: The classification output module processes the semantic enhancement features through a global average pooling layer and a fully connected layer to output the disease classification results of the medical image.

5. A medical image classification method based on a structure-aware state-space model, applied to a medical image classification system based on any one of the structure-aware state-space models described in claims 1-4, characterized in that, Includes the following steps: S1. Preprocess the input medical image data; S2. Input the preprocessed medical image data into the input adaptation module for feature projection processing, and then pass the processed feature map to the multi-scale feature fusion module for multi-scale feature extraction and fusion. S3. Input the fused multi-scale features into the structure-aware state space modeling module. Extract edge features from medical images through edge-aware convolution to generate an edge response map. Combine the edge response map with the edge features and structural features, and use the edge response map as a weighting factor to perform weighted processing on the edge features and structural features, thereby incorporating structural information into the state update process and generating structural features. S4. Adaptively adjust the channel weights based on the features to enhance disease-related semantic features, suppress background noise, and output semantically enhanced features; S5. After semantic enhancement features are processed by a global average pooling layer and a fully connected layer, the output is the disease classification result of the medical image. The structure-aware state-space modeling module extracts edge features from medical images through edge-aware convolution to generate structural features, specifically including: Edge and structural features are extracted from medical images using edge-aware convolutional layers and the SiLU activation function. These edge and structural features are then input into the update equation of the structure-aware state-space modeling module. The update equation is as follows: For structural features, For edge features, A is the state transition moment, B is the weight matrix, C is a convolution operation, ΔA is the change in the state transition matrix A, and I is the identity matrix; The dynamic channel gating module performs semantic filtering and weighting through a dynamic gating mechanism, and outputs semantically enhanced features, specifically including: The dynamic channel gating module adaptively adjusts the channel weights based on features, enhances disease-related semantic features, suppresses background noise, and outputs semantically enhanced features. Its calculation method is as follows: Where x is the input vector, Weight matrix, and Responsible for dimensional increase / transformation, Responsible for projecting dimensions back to their original size. (·) is the activation function, ⊙ represents element-wise multiplication, σ is the Sigmoid function, and y is the output vector.

Images