A medical image classification method and system based on asymmetric feature interaction

Abstract

The application discloses a medical image classification method and system based on asymmetric feature interaction, which firstly carries out pretreatment on medical image data obtained from a medical imaging device to obtain standardized image input data. Secondly, the image input data is sent to a convolution projection layer to generate low-level feature representation, and the low-level feature representation is sequentially subjected to stepwise down-sampling and feature extraction through multiple hierarchical mixing blocks for output of enhanced feature maps of different scales. Finally, the enhanced feature maps output by each mixing block in all mixing blocks are read and spliced to form a unified feature representation input into a multi-layer perception classification head to output a medical image classification prediction result. The application realizes efficient and accurate medical image feature extraction in view of inherent challenges such as structural heterogeneity, large lesion scale difference and low contrast in medical images.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and computer vision technology, and specifically relates to a visual coding method and system for medical image classification. Background Technology

[0002] With the development of deep learning, Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have become mainstream tools for medical image analysis: CNN-type models (such as ResNet and DenseNet) extract features through local convolutions and have good local modeling capabilities, but they struggle to capture global contextual information and have limited ability to model long-distance dependencies. ViT-type models (such as Swin Transformer and MedViT) model global dependencies through self-attention mechanisms and can effectively capture long-distance features, but in medical images, they suffer from low feature interaction efficiency and high computational cost when dealing with asymmetrical anatomical structures and lesions with large scale variations. Hybrid models (such as CBAM and Coordinate Attention) introduce attention mechanisms to enhance feature representation, but they are mostly symmetrical structures, failing to effectively decouple spatial and semantic features, and lack sensitivity to small-scale lesions.

[0003] In recent years, to overcome the aforementioned limitations, researchers have improved medical image visual encoders from multiple perspectives. One type of work introduces attention mechanisms into convolutional networks, enhancing the model's ability to filter key features by introducing adaptive weighting in the channel and spatial dimensions. However, most of these attention modules adopt symmetric structures, applying a uniform weighting strategy to spatial and semantic information, without fully considering the high degree of asymmetry in the distribution of spatial and semantic features in medical images. Another type of work is based on transformer architecture, improving multi-scale feature modeling capabilities through a hybrid convolution-attention design and a hierarchical token interaction mechanism.

[0004] Despite the progress made by the above methods on specific tasks, existing models still generally suffer from the following shortcomings: most models rely on symmetric or standardized attention mechanisms, ignoring the asymmetric nature of spatial and semantic feature distributions in medical images, resulting in low feature interaction efficiency; existing architectures usually prioritize the aggregation of global information and lack mechanisms to explicitly capture small lesions or subtle local structures, which are crucial for accurate diagnosis; some transformer-based methods, although having strong modeling capabilities, have high computational complexity and slow inference speed, making it difficult to meet the real-time and lightweight requirements of actual clinical deployments.

[0005] In summary, existing medical image classification technologies urgently need a visual coding scheme that can achieve efficient asymmetric feature interaction, balance global semantic perception and local detail enhancement, and possess lightweight computational characteristics. Summary of the Invention

[0006] This invention addresses the aforementioned problems by proposing a medical image classification method and system based on asymmetric feature interaction, particularly suitable for processing medical images with complex structures, large lesion scale variations, and low contrast, such as CT, MRI, pathological sections, and fundus images. This invention solves the shortcomings of existing technologies on three levels: First, in terms of feature organization, this invention abandons the traditional encoder paradigm of treating all feature channels equally and introduces a dynamic asymmetric channel separation strategy based on data complexity. This strategy adaptively divides feature channels into spatially dominant branches and semantically dominant branches by learning the global complexity factor of the input image. Second, in terms of feature interaction, this invention designs a spatial-semantic guided cross-attention mechanism to establish a bidirectional information flow between spatial and semantic branches. This cross-guided mechanism fundamentally differs from traditional self-attention or symmetric attention, achieving efficient fusion of complementary features within the same modality. Third, in terms of feature enhancement, this invention introduces a context enhancement module based on deformable receptive fields. By extracting high-frequency detail features and global context features in parallel and using deformable convolution to adaptively match irregular lesion morphologies, this enhances the model's sensitivity to small, low-contrast lesion regions.

[0007] In one aspect, this invention provides a medical image classification method based on asymmetric feature interaction, comprising the following steps: Step 1: Preprocess the medical image data acquired from the medical imaging equipment to obtain standardized image input data.

[0008] Step 2: The image input data is fed into a convolutional projection layer to generate a low-level feature representation. This low-level feature representation is then progressively downsampled and feature extracted through multiple hierarchical blending blocks to output enhanced feature maps at different scales. Each blending block contains convolutional layers and feature interaction blocks, used for extracting local features and fusing and refining cross-branch features. After convolutional projection, the input image passes through multiple blending blocks, ultimately using the multi-scale enhanced feature maps for subsequent classification head medical image classification.

[0009] Step 3: Read the enhanced feature maps from each of the mixed blocks and stitch them together to form a unified feature representation. Input the unified feature representation into the multilayer perceptron classification head and output the medical image classification prediction result.

[0010] Furthermore, the implementation of the feature interaction block is as follows: Step 2.1: For the feature map in each input feature interaction block, first estimate the channel complexity, and then obtain the spatial features and semantic features by weighted calculation based on the channel complexity.

[0011] Step 2.2: Spatial features and semantic features are merged into global features through bidirectional weighted fusion.

[0012] Step 2.3: Extract two types of features in parallel from the global features using a deformable receptive field mechanism. After concatenating the two types of features, generate enhancement weights through a lightweight attention mechanism, and obtain the enhanced feature map output by the feature interaction block at this scale through residual connections.

[0013] In another aspect, the present invention also provides a medical image classification system based on asymmetric feature interaction, comprising the following modules: The medical image module is used to preprocess medical image data acquired from medical imaging equipment to obtain standardized image input data.

[0014] The asymmetric feature interaction module is used to feed image input data into the convolutional projection layer to generate low-level feature representations. The low-level feature representations are then progressively downsampled and feature extracted through multiple hierarchical mixing blocks to output enhanced feature maps at different scales.

[0015] The medical image classification output module is used to stitch together the enhanced feature maps output from each of the mixed blocks to form a unified feature representation, which is then input into the multilayer perceptron classification head to output the medical image classification prediction result.

[0016] Beneficial effects of the invention This invention is specifically designed for medical image classification tasks. Addressing the inherent challenges of medical images, such as structural heterogeneity, large lesion scale variations, and low contrast, it achieves efficient and accurate medical image feature extraction through an innovative asymmetric feature interaction mechanism and a spectral context enhancement module. Compared to existing technologies, this invention improves classification accuracy while balancing high performance and low computational overhead. This lightweight design allows for efficient deployment in computationally limited clinical environments, meeting the needs of real-time or near-real-time diagnosis and demonstrating promising practical application prospects. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 This is a comparison of the attention heatmap of the present invention with other baseline models. Detailed Implementation

[0018] See the implementation flowchart. Figure 1 In one aspect, this invention provides a medical image classification method (EchoNet) based on asymmetric feature interactions, comprising the following steps: Step 1: Data Preprocessing and Image Input Construction. For DICOM format medical image data acquired from medical imaging equipment (such as CT, MRI, PET, etc.), the pixel matrix is ​​first read as the raw image data. Subsequently, according to the specific imaging modality and task requirements (such as the window width and window level settings for brain CT), the raw image is grayscale normalized and uniformly scaled to a fixed resolution (such as 28×28 or 224×224), finally obtaining standardized image input data.

[0019] Step 2: Preliminary Feature Extraction and Downsampling. The image input data obtained in Step 1 is fed into a convolutional projection layer to generate a low-level feature representation. Subsequently, the low-level feature representation is progressively downsampled and feature extracted through multiple hierarchical hybrid blocks. Each hybrid block contains a convolutional layer and a feature interaction block. Each internal feature interaction block consists of the processes described in Steps 3, 4, and 5, and is used to output enhanced feature maps of different scales for final classification.

[0020] Step 3: Asymmetric Channel Separation (ACS). For the feature map in each input feature interaction block, the content complexity estimator is first fed into the channel complexity estimator to estimate the channel complexity. Then, based on the channel complexity, spatial features and semantic features are obtained by dynamically calculating along the channel dimension through weighted calculation. This step realizes the division of labor and decoupling between spatial features and semantic features.

[0021] Step 4: Spatial-Semantic Guided Cross-Attention Fusion (SSGCA). The spatial and semantic features obtained in Step 3 are fused through bidirectional weighted fusion to obtain enhanced features, which are then merged into global features.

[0022] Step 5: Variable Receptive Field Context Enhancement (CEM). The global features obtained in Step 4 are input into the context enhancement module. Two types of features are extracted in parallel through a deformable receptive field mechanism. After concatenation, enhancement weights are generated through a lightweight attention mechanism, and the enhanced feature map output by the feature interaction block at this scale is obtained through residual connections.

[0023] Step 6: Multi-scale feature aggregation and classification output. The enhanced feature maps from each of the mixing blocks are read and concatenated to form a unified feature representation. This feature is then input into the multilayer perceptron classification head to output the final image classification prediction result.

[0024] Furthermore, step 3 is specifically implemented as follows: for the input feature map... First, input the content complexity estimator to obtain the channel complexity. The content complexity estimator consists of global pooling and continuous lightweight subnetworks.

[0025] Based on channel complexity Dynamic channel allocation is achieved through soft mask weighting, where For temperature parameters, Channel index: Indicates spatial separation weights. This indicates the channel weight.

[0026] Two features are obtained through weighted calculation: Spatial features .

[0027] semantic features .

[0028] In existing technologies, channel separation is mostly static or based on predefined rules. This separation strategy is based on a dynamic channel allocation mechanism of image content complexity, which enables the model to adaptively adjust the resource allocation of spatial and semantic features, improves the modeling ability for lesions of different scales and contrasts, and reduces redundant computation.

[0029] Furthermore, step 4 is specifically implemented by optimizing semantic feature learning through the spatial context provided by spatial features, while allowing the semantic response of semantic features to guide channel reweighting in spatial representation, thus bidirectionally guiding fusion: For activation function, For batch standardization, for convolution, Global average pooling, for Depth convolution.

[0030] pass and The global features are obtained by addition. : This spatial-semantic bidirectional dependency mechanism enables the model to achieve dynamic feature fusion across scales and semantic levels, significantly improving its discriminative ability in low-contrast and structurally asymmetric medical images. Furthermore, since the entire SSGCA mechanism is based on lightweight convolutions and global statistical features, its computational cost is significantly reduced compared to traditional self-attention mechanisms, while still maintaining strong feature interaction representation efficiency.

[0031] Furthermore, step 5 is specifically implemented by extracting two types of features in parallel through a deformable receptive field mechanism: High-frequency detail features Global context features for Activation function For variable convolution; and This is a learnable offset.

[0032] The mathematical form of variable convolution is: in For output position, For the preset first The initial offset of each sampling point The learned offset, The weights are used for the convolution kernel, and non-integer coordinates are sampled using bilinear interpolation.

[0033] High-frequency detail features and global context features After concatenation, enhanced weights are generated through a lightweight attention mechanism, and enhanced feature maps are obtained from the feature interaction blocks at this scale through residual connections.

[0034] The deformable receptive field context enhancement module can adaptively deform to match irregular lesion morphology. Offset field co-learning ensures consistency of multi-scale features, significantly improving the ability to model complex anatomical structures, and has significant geometrical adaptive innovation.

[0035] In another aspect, the present invention provides a medical image classification system based on asymmetric feature interaction for implementing the aforementioned medical image classification method, comprising a medical image module, an asymmetric feature interaction module, and a medical image classification output module. The medical image module is used to preprocess medical image data acquired from medical imaging equipment to obtain standardized image input data.

[0036] The asymmetric feature interaction module is used to feed the image input data into the convolutional projection layer to generate a low-level feature representation. The low-level feature representation is then progressively downsampled and feature extracted through multiple hierarchical mixing blocks to output enhanced feature maps at different scales.

[0037] The medical image classification output module is used to stitch together the enhanced feature maps output from each of the mixed blocks to form a unified feature representation, which is then input into the multilayer perceptron classification head to output the medical image classification prediction result.

[0038] Experimental verification: To comprehensively evaluate the effectiveness and generalization ability of the proposed feature interaction block module, this invention conducted extensive experiments on multiple medical image benchmark datasets and several task-specific datasets. These datasets cover a variety of image modalities and pathological states, providing a solid foundation for evaluating the module's performance and applicability. By testing the feature interaction block on heterogeneous data sources, the aim is to demonstrate its generalization ability across different tasks and to prove the diversity of its architectural design. This comprehensive evaluation ensures the statistical reliability of the results while highlighting the potential for widespread application of feature interaction blocks in medical image analysis.

[0039] First, the proposed EchoNet was systematically evaluated using the MedMNIST benchmark suite. MedMNIST is a standardized and lightweight medical imaging benchmark designed to efficiently evaluate the performance of visual models in medical applications. It contains twelve 2D classification subsets, covering a variety of imaging modalities and diagnostic tasks. The MedMNIST benchmark provides a unified platform for fair comparisons across different tasks, ensuring both task diversity and result consistency. All samples were grayscale normalized and adjusted to a uniform resolution. We followed standard training, validation, and test data splitting. Model optimization was performed only on the training set, with the test set reserved for final performance evaluation. To ensure fairness and reproducibility, no additional spatial augmentation or resolution upscaling techniques were applied.

[0040] All experiments were conducted on a standardized computing platform to ensure consistency and reproducibility. The system was based on an Ubuntu 20.04 server operating system, equipped with an NVIDIA 3090Ti GPU. The software environment included CUDA 12.8 and PyTorch 2.8.0, providing the necessary tools for model implementation and training. The training process was configured with a learning rate of 0.0001 and a training duration of 200 epochs. Parameter optimization employed the Adam optimizer, using cross-entropy loss to guide model learning. These settings were designed to achieve stable convergence and robust performance across all datasets.

[0041] First, the invention was evaluated on the MedMNIST benchmark suite, and the results are detailed in Tables 1 and 2. Overall, the invention achieved superior or near-superior performance in most tasks. Specifically, on the PathMNIST, DermaMNIST, OCTMNIST, and PneumoniaMNIST datasets, EchoNet achieved classification accuracies of 0.8534, 0.7486, 0.7830, and 0.8830, respectively, surpassing ResNet50 and the MedVi series models. Notably, compared to suboptimal methods, EchoNet improved the accuracy on OCTMNIST by approximately 2.0%, highlighting the ability of the feature interaction block module to effectively model detailed structural representations.

[0042] EchoNet also demonstrates consistent and stable performance on other datasets such as BreastMNIST, BloodMNIST, TissueMNIST, OrganCMNIST, and OrganSMNIST, with average accuracy improvements ranging from 0.1% to 2.0% and AUC improvements ranging from 0.1% to 2.8%. Particularly noteworthy is EchoNet's AUC of 0.9223 on BreastMNIST, the highest among all compared methods. This invention maintains good computational efficiency, with an inference time of 166.22 milliseconds. This lightweight design contrasts sharply with several Transformer-based models, showcasing the balance between efficiency and performance achieved by EchoNet.

[0043] These results collectively confirm that the Feature Interaction Block Module (SSGCA) enhances information flow between branches, while SCEM particularly strengthens the response of multi-scale structures. Therefore, this invention provides more accurate and robust representations across different image modalities, demonstrating its strong adaptability and discriminative capabilities in cross-modal contexts.

[0044] To gain a deeper understanding of the advantages of this invention and to qualitatively assess the accuracy of its region of interest, Grad-CAM technology was used to generate attention heatmaps for different models during the inference process. Figure 2 The presentation shows a comparison of attention heatmaps between the present invention and other baseline models (such as ResNet50 and MedVit) on multiple MedMNIST datasets.

[0045] Analysis shows that, compared to the contrasting model, the attention map generated by this invention is more focused on the lesion area, especially when dealing with small targets or low-contrast lesions. For example, in the OCTMNIST and PneumoniaMNIST tasks, this invention can more accurately locate retinal effusion or lung infection areas, while the attention of the contrasting model is relatively scattered and easily interfered with by background noise or other irrelevant anatomical structures.

[0046] In contrast, the attention patterns generated by this invention are more precise and focused, effectively highlighting key local lesion areas while maintaining robust global context awareness. This advantage is particularly evident in structurally complex or low-contrast images, where the high-response areas of this invention more closely match the actual lesion locations.

[0047] These visualizations highlight the contributions of the core modules of this invention. The SSGCA module enhances efficient information exchange across branches, while the SCEM module improves the model's sensitivity to subtle lesions and boundary features. Together, these modules enable this invention to better capture discriminative regions in medical images, providing strong interpretability for its performance improvements and demonstrating its superiority in spatial attention modeling. Table 1 below shows the comparison results of different models on the first subset of the MedMNIST benchmark (including PathMNIST, OCTMNIST, and PneumoniaMNIST). Evaluation metrics include classification accuracy (ACC) and area under the ROC curve (AUC). Inference time is the time (milliseconds) required to process one batch in the evaluation mode without gradient computation. Table 2 shows the comparison results of different models on the second subset of the MedMNIST benchmark (including BreastMNIST, BloodMNIST, OrganCMNIST, and OrganSMNIST). Evaluation metrics are ACC and AUC.

[0048] Table 1

[0049] Table 2

Claims

1. A medical image classification method based on asymmetric feature interaction, characterized in that, Includes the following steps: Step 1: Preprocess the medical image data acquired from the medical imaging equipment to obtain standardized image input data; Step 2: Feed the image input data into the convolutional projection layer to generate low-level feature representations. The low-level feature representations are then progressively downsampled and feature extracted through multiple hierarchical mixing blocks to output enhanced feature maps at different scales. Step 3: Read the enhanced feature maps from each of the mixed blocks and stitch them together to form a unified feature representation. Input the unified feature representation into the multilayer perceptron classification head and output the medical image classification prediction result.

2. The medical image classification method based on asymmetric feature interaction according to claim 1, characterized in that, The hybrid block comprises convolutional layers and feature interaction blocks, the implementation of which is as follows: Step 2.1: For the feature map in each input feature interaction block, first estimate the channel complexity, and then obtain the spatial features and semantic features by weighted calculation based on the channel complexity; Step 2.2: For spatial features and semantic features, merge them into global features through bidirectional weighted fusion; Step 2.3: Extract two types of features in parallel from the global features using a deformable receptive field mechanism. After concatenating the two types of features, generate enhancement weights through a lightweight attention mechanism, and obtain the enhanced feature map output by the feature interaction block at this scale through residual connections.

3. The medical image classification method based on asymmetric feature interaction according to claim 2, characterized in that, Step 2.1 is specifically implemented as follows: after the input image undergoes convolutional projection, for the input feature map... First, input the content complexity estimator to obtain the channel complexity. ; Based on channel complexity Dynamic channel allocation is achieved through soft mask weighting, where For temperature parameters, Channel index: ； in, Indicates spatial separation weights. The channel weights are represented; then, two features are obtained through weighted calculation: Spatial features ; semantic features .

4. The medical image classification method based on asymmetric feature interaction according to claim 3, characterized in that, The content complexity estimator consists of global pooling and continuous lightweight subnetworks.

5. The medical image classification method based on asymmetric feature interaction according to claim 4, characterized in that, Step 2.2 is specifically implemented as follows: Semantic feature learning is optimized through the spatial context provided by spatial features, while the semantic response of the semantic features guides channel reweighting in the spatial representation, thus bidirectionally guiding the fusion. For activation function, For batch standardization, for convolution, Global average pooling, For depthwise convolution; pass and The global features are obtained by addition. .

6. The medical image classification method based on asymmetric feature interaction according to claim 5, characterized in that, Step 2.3 is specifically implemented as follows: two types of features are extracted in parallel using a deformable receptive field mechanism: High-frequency detail features ; Global context features ; in, for Activation function For variable convolution; and The learnable offset; High-frequency detail features and global context features After concatenation, enhanced weights are generated through a lightweight attention mechanism, and enhanced feature maps are obtained from the feature interaction blocks at this scale through residual connections.

7. A medical image classification system based on asymmetric feature interaction, used to implement the medical image classification method according to any one of claims 1 to 6, characterized in that, Includes the following modules: The medical image module is used to preprocess medical image data acquired from medical imaging equipment to obtain standardized image input data; The asymmetric feature interaction module is used to feed the image input data into the convolutional projection layer to generate low-level feature representations. The low-level feature representations are then progressively downsampled and feature extracted through multiple hierarchical mixing blocks to output enhanced feature maps at different scales. The medical image classification output module is used to stitch together the enhanced feature maps output from each of the mixed blocks to form a unified feature representation, which is then input into the multilayer perceptron classification head to output the medical image classification prediction result.

Images