Medical image classification method and system based on multi-scale spatial state modeling

Abstract

The invention discloses a medical image classification method and system based on multi-scale spatial state modeling, and the method comprises the steps: firstly dividing an input medical image into a plurality of non-overlapping image blocks, and mapping the non-overlapping image blocks to a feature space through a learnable linear projection layer to obtain an initial feature map; then, multiple layers of stacked MS-SMamba blocks are used for carrying out layer-by-layer feature extraction, each MS-SMamba block comprises a main branch, an auxiliary branch, a dynamic gating fusion network, a residual error connection unit and a feedforward network, and long-range dependency relation capture and multi-scale feature fusion are achieved; and finally, processing the last-layer output feature map through a global feature aggregation and classification module, generating a global feature vector, and outputting a classification result. According to the method, the capturing capability of complex pathological features in the medical image is improved, the calculation efficiency and clinical applicability are improved, and the method is suitable for scenes such as disease screening and auxiliary decision making in medical image diagnosis.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis technology, specifically to a medical image classification method and system based on multi-scale spatial state modeling. Background Technology

[0002] Medical image classification, a core component of disease diagnosis, is undergoing a transformation from manual image interpretation to intelligent analysis. With the increasing resolution of medical imaging equipment (such as 3D ultrasound and high-resolution CT) and the widespread adoption of multimodal imaging technologies, medical image data exhibits the following characteristics:

[0003] I. Data Complexity:

[0004] Lesion heterogeneity: The same type of disease exhibits significant morphological differences at different stages of development (such as the morphological evolution of tumors from microcalcification to invasive growth), while different diseases often have similar imaging features (such as the overlap of ground-glass opacities in pneumonia and tuberculosis).

[0005] Structural hierarchy: The human anatomy presents a multi-level connection between organs, tissues and cells (such as the anatomical coupling between liver tumors and intrahepatic blood vessels), and lesions often exhibit complex characteristics such as small size (diameter <2mm) and weak boundaries (such as early liver fibrosis).

[0006] Modal diversity: A single modality is insufficient to fully characterize pathological information (e.g., MRI provides soft tissue contrast, PET reflects metabolic activity), and multimodal fusion has become the key to improving diagnostic efficiency.

[0007] II. Structural Specificities:

[0008] Diagnostic efficiency bottleneck: A WHO report shows that the global annual growth rate of medical imaging data has reached 30%.

[0009] Diagnostic accuracy requirements: Taking breast cancer screening as an example, the US FDA requires the false negative rate of the screening system to be less than 1%, while the sensitivity of traditional CAD systems in dense breasts is less than 70%.

[0010] Real-time constraints: Clinical scenarios such as intraoperative navigation require model inference latency of less than 10ms, and existing methods are difficult to meet the stringent real-time requirements.

[0011] Urgent clinical need: WHO reports show that global medical imaging data is growing at an annual rate of 30%, creating an urgent need for efficient and accurate automated classification tools.

[0012] Current mainstream medical image classification technologies can be divided into three categories: solutions based on convolutional neural networks (CNN), visual Transformer (ViT), and state-space model (SSM), but all of them have significant limitations.

[0013] With the development of deep learning technology, Convolutional Neural Networks (CNNs) and Visual Transformers (ViTs) have become mainstream methods, but they still have significant limitations in medical image analysis:

[0014] I. A Convolutional Neural Network (CNN)-based approach extracts multi-scale features through parallel multi-dilation rate convolutional branches, captures multi-scale contextual features through dilated convolutions with different sampling rates, and enhances detail recovery through feature map concatenation. However, parallel convolutional branches with a fixed dilation rate cannot adapt to changes in lesion scale, traditional skip connections suffer from feature alignment errors, and multi-scale feature fusion lacks content-adaptive weight allocation. Furthermore, the number of parameters in a 3D CNN increases cubically with dimensionality. Therefore, while parallel dilated convolutional branches can capture multi-scale features, their fixed receptive field design leads to insufficient sensitivity for small target detection, and the number of model parameters increases exponentially with depth.

[0015] II. The Visual Transformer (ViT)-based approach achieves global dependency modeling through patch-based self-attention, introduces a moving window mechanism to balance computational efficiency and global perception, and employs a contrastive learning strategy to improve cross-center generalization ability. However, patch partitioning leads to the breakage of key anatomical structures, and positional encoding struggles to adapt to the complex topological relationships in medical images. ViT's long-sequence modeling results in a surge in memory consumption, making it difficult to deploy on embedded devices, and it lacks robustness to unstructured noise such as speckle noise in ultrasound images and artifacts in MRI. Therefore, while ViT's global self-attention mechanism performs excellently in natural images, the small target characteristics of medical images lead to a sparse distribution of attention weights, resulting in decreased practical computational efficiency.

[0016] Third, the State-Space Model (SSM)-based approach is the first to introduce the Mamba architecture into medical image segmentation tasks. It achieves efficient 3D medical image segmentation through variant state-space modeling, integrating window attention and state-space scanning strategies. However, existing SSM scanning strategies (such as four-way scanning) artificially distort spatial adjacency relationships, and existing SSM models rely on fixed scan steps and cannot adaptively adjust the perception range, resulting in inference latency in 3D medical image segmentation, which is insufficient to meet the real-time requirements of intraoperative navigation. Therefore, although performance is improved by expanding the state dimension, the computational complexity of 3D convolution leads to increased inference latency, making it difficult to meet clinical real-time requirements.

[0017] Therefore, how to provide a medical image classification method and system that can achieve a balance between multi-scale feature extraction, spatial structure preservation, and computational efficiency optimization is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0018] In view of this, the present invention provides a medical image classification method and system based on multi-scale spatial state modeling. By integrating multi-scale spatial perception mechanism and state space modeling architecture, it improves the ability to capture complex pathological features in medical images and is applicable to scenarios such as disease screening and decision support in medical image diagnosis.

[0019] To achieve the above objectives, the present invention adopts the following technical solution:

[0020] This invention first proposes a medical image classification method based on multi-scale spatial state modeling, comprising the following steps:

[0021] S1: Divide the input medical image into multiple non-overlapping image patches; map each image patch to a feature space of a preset dimension through a learnable linear projection layer to obtain an initial feature map;

[0022] S2: The initial feature map is input into a multi-layered stacked MS-SMamba block for layer-by-layer feature extraction; wherein each MS-SMamba block includes:

[0023] The main branch is used to perform selective state-space modeling scanning through the state-space model, capture long-range dependencies, and output the first feature.

[0024] An auxiliary branch is used to extract multi-scale features from the first feature, generate content-adaptive dynamic weights based on the global statistical information of the first feature, perform weighted fusion of the multi-scale features, and output the second feature.

[0025] A dynamic gated fusion network is used to generate a gated weight map based on the first feature and the second feature, and to adaptively weight and fuse the first feature and the second feature based on the gated weight map to obtain enhanced features;

[0026] Residual connection units and feedforward networks are used to perform residual connection processing on the enhanced features to obtain the output feature map of the current layer;

[0027] S3: Perform global feature aggregation on the output feature map of the last layer MS-SMamba block to obtain a global feature vector; classify and output the classification result of the medical image based on the global feature vector.

[0028] Preferably, after mapping each image patch to a feature space of a preset dimension through a learnable linear projection layer in step S1 to obtain the feature embedding vector, the following steps are also included:

[0029] By using depthwise separable convolution kernels, local spatial relationships are perceived through convolution based on the feature embedding vectors, and positional codes are dynamically generated and fused, so that the output initial feature map includes spatial positional information.

[0030] Preferably, the main branch is used to perform the following steps:

[0031] The input two-dimensional initial feature map is scanned in four directions by a four-way scanner and converted into a one-dimensional sequence.

[0032] The one-dimensional sequence is input into a selective state-space model for processing to capture long-range dependencies;

[0033] The output of the selective state-space model is remapped back to two-dimensional space through an expanded convolutional weight reconstructor.

[0034] Preferably, the step of the auxiliary branch performing multi-scale feature extraction on the first feature includes:

[0035] Multiple dilation convolution operations with different dilation rates are performed in parallel on the first feature to extract features at different scales.

[0036] Preferably, the step of generating content-adaptive dynamic weights based on the global statistical information of the first feature, weighting and fusing multi-scale features, and outputting the second feature includes:

[0037] The global context feature vector of the first feature is extracted by global average pooling;

[0038] The global context feature vector is mapped to the same dimension as the number of branches in the dilated convolution through a fully connected layer.

[0039] Normalization is performed using the Softmax function to generate dynamic weight coefficients for each dilated convolution branch.

[0040] The corresponding multi-scale spatial features are weighted and fused using the dynamic weighting coefficients to obtain the second feature.

[0041] Preferably, the adaptive weighted fusion step of the fusion gating network includes:

[0042] The first feature and the second feature are concatenated along the channel dimension to generate a joint feature;

[0043] The joint features are input into a convolutional bottleneck network, and an activation function is applied to the output of the bottleneck network to generate a pixel-level gated weight map γ∈[0,1].

[0044] Based on the gating weight map, the enhanced feature h_final = γ⊙h_main + (1-γ)⊙h_fused is fused, where h_main is the first feature, h_fused is the second feature, and ⊙ is the Hadamard product.

[0045] Preferably, the operation of global feature aggregation of the output feature map of the last layer MS-SMamba block in S3 includes processing the output feature map through a global average pooling layer and then inputting it into a fully connected layer to output the category probability of the medical image.

[0046] The present invention further provides a medical image classification system based on multi-scale spatial state modeling, according to the aforementioned medical image classification method based on multi-scale spatial state modeling, comprising:

[0047] The input preprocessing and feature embedding module receives the input medical image and divides it into multiple non-overlapping image patches; it then maps each image patch to a feature space of a preset dimension through a learnable linear projection layer to obtain an initial feature map.

[0048] A multi-layered stacked MS-SMamba block module is used to perform layer-by-layer feature extraction on the initial feature map. Each MS-SMamba block includes:

[0049] The main branch is used to perform selective state-space modeling scanning through the state-space model, capture long-range dependencies, and output the first feature.

[0050] An auxiliary branch is used to extract multi-scale features from the first feature, generate content-adaptive dynamic weights based on the global statistical information of the first feature, perform weighted fusion of the multi-scale features, and output the second feature.

[0051] A dynamic gated fusion network is used to generate a gated weight map based on the first feature and the second feature, and to adaptively weight and fuse the first feature and the second feature based on the gated weight map to obtain enhanced features;

[0052] Residual connection units and feedforward networks are used to perform residual connection processing on the enhanced features to obtain the output feature map of the current layer;

[0053] The global feature aggregation and classification module is used to perform global feature aggregation on the output feature map of the last layer MS-SMamba block to obtain a global feature vector; and to perform classification based on the global feature vector to output the classification result of the medical image.

[0054] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a medical image classification method and system based on multi-scale spatial state modeling. Through an innovative spatial-scale joint modeling mechanism and dynamic fusion architecture, the following beneficial effects are achieved:

[0055] 1. Accurate capture of multi-scale pathological features. By introducing a multi-scale structure perception module and a cross-scale dilated convolution array, the model can adaptively match pathological features at different scales:

[0056] ① The detection sensitivity for tiny lesions with a diameter of <2mm has been improved;

[0057] ② Improved completeness of complex organ structure modeling;

[0058] ③ Multi-scale feature fusion effectively suppresses intra-class differences.

[0059] 2. Breakthrough in Spatial-Sequence Modeling Efficiency. A pioneering dynamic gating fusion mechanism and hardware-friendly scanning strategy achieve dual optimization of computational efficiency and representational capability:

[0060] ① The computational complexity of a single forward pass remains linearly O(HW), which is three orders of magnitude lower than that of ViT;

[0061] ② Achieved an inference throughput of 1540 img / s on the NVIDIA 4090D GPU, a significant speedup compared to 2D Mamba;

[0062] ③ The number of parameters is compressed to 29M, which is significantly less than that of the traditional ViT, meeting the deployment requirements of embedded devices.

[0063] 3. Significantly enhanced adaptability to medical scenarios. Through enhanced medical interpretability design and noise robustness optimization, clinical applicability is greatly improved:

[0064] ①Saliency analysis showed increased attention to the lesion area;

[0065] ②Increased tolerance to speckle noise in ultrasound images;

[0066] ③ Supports multi-resolution input from 224×224 to 512×512, with a constant number of parameters and linear growth of FLOPs.

[0067] 4. Innovative Long-Range Dependency Modeling Capabilities. Based on state-space scanning strategy optimization and content-adaptive weight allocation, it breaks through the spatiotemporal modeling limitations of traditional models:

[0068] ① The accuracy of modeling the association between anatomical structures between organs is improved by capturing global dependencies based on cross-scale interaction modules;

[0069] ② The context awareness range for small targets is expanded, which is effectively improved compared to fixed scanning strategies;

[0070] ③ The memory usage for modeling long sequences was reduced through depthwise separable convolution optimization.

[0071] 5. Enhanced feasibility for clinical deployment. Through lightweight gating mechanisms and adaptive computation scheduling, it meets the stringent requirements of medical scenarios:

[0072] ① The inference latency is controlled at 5.8ms, and the batch processing is 128, which meets the requirements for real-time diagnosis;

[0073] ② Reduced power consumption when deployed on mobile devices. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0075] Figure 1 The overall architecture diagram of the medical image classification method model based on multi-scale spatial state modeling provided in the embodiments of the present invention is shown below.

[0076] Figure 2 A schematic diagram of the MS-SMamba block in an auxiliary branch provided in an embodiment of the present invention;

[0077] Figure 3 A schematic diagram of a cross-scale interaction module based on multi-scale dilated convolution provided in an embodiment of the present invention;

[0078] Figure 4 A schematic diagram illustrating the dynamic weighting mechanism provided in this embodiment of the invention;

[0079] Figure 5 The comparison results of feature heatmaps provided in the embodiments of the present invention. Detailed Implementation

[0080] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0081] The first aspect of this invention discloses a medical image classification method based on multi-scale spatial state modeling, comprising the following steps:

[0082] S1: The input medical image is divided into multiple non-overlapping image patches; each image patch is mapped to a feature space of a preset dimension through a learnable linear projection layer to obtain an initial feature map. S1 is used to complete the transformation from image to sequence features and establish a spatially aware semantic representation.

[0083] In one embodiment, after mapping each image patch to a feature space of a preset dimension through a learnable linear projection layer in S1 to obtain the feature embedding vector, the following steps are also included:

[0084] By utilizing depthwise separable convolution kernels, local spatial relationships are perceived through convolution of feature embedding vectors, and positional codes are dynamically generated and fused, so that the initial feature map output includes spatial positional information.

[0085] like Figure 1 As shown, a 4×4 non-overlapping block strategy is used to transform the input image H×W×C into a (H / 4)×(W / 4)×16C feature tensor. This tensor is then mapped to a D-dimensional feature space through a learnable linear projection layer.

[0086] The specific execution process of S1 is as follows:

[0087] ① The input medical image (B×H×W×C) is first segmented into 4×4 pixel non-overlapping image blocks (B×(H / 4)×(W / 4)×16C) by patch partitioning units. Each block is flattened into a 16C-dimensional feature vector, where B is the batch size, representing the number of image samples processed in a single forward computation. During deep learning training / inference, batch processing is used to improve computational parallelism and efficiency. H is the height of the input medical image (in pixels), W is the width of the input medical image (in pixels), and C is the number of channels of the input image (Channel Depth), representing the color or modal dimension of the image.

[0088] ② The linear projection layer maps each image patch to a D-dimensional feature space (B×(H / 4)×(W / 4)×D):

[0089]

[0090] in It is the flattened feature tensor (4×4 non-overlapping block) output by the image patch partitioning unit. It is a learnable projection matrix that maps 16C-dimensional features to a D-dimensional hidden space. D is the hidden dimension of the features, that is, the number of channels of the features inside the model. It defines the representational power of the hidden layer of the model. The larger the value, the stronger the representational power (but the higher the computational cost). It is the projected feature tensor. Its function is to convert the original pixel block into high-dimensional semantic features, providing a foundation for subsequent sequence modeling.

[0091] ③ The CPE module dynamically generates a location code (B×(H / 4)×(W / 4)×D) based on the input features:

[0092]

[0093] Where Z is the feature tensor after linear projection. It is a depthwise separable convolution kernel (kernel size k×k, usually k=3). It is a dynamically generated positional encoding. Its function is to perceive local spatial relationships through convolution, inject positional sensitivity into features, and solve the problem of spatial structure distortion caused by image patch serialization.

[0094] S2: Input the initial feature map into a multi-layered stacked MS-SMamba block for layer-by-layer feature extraction; where each MS-SMamba block includes:

[0095] The main branch is used to perform selective state-space modeling scanning through the state-space model, capture long-range dependencies, and output the first feature.

[0096] The auxiliary branch is used to extract multi-scale features from the first feature and generate content-adaptive dynamic weights based on the global statistical information of the first feature. It then performs weighted fusion of the multi-scale features and outputs the second feature.

[0097] A dynamic gated fusion network is used to generate a gated weight map based on a first feature and a second feature, and then adaptively weightedly fuses the first feature and the second feature based on the gated weight map to obtain the enhanced feature.

[0098] Residual connection units and feedforward networks are used to perform residual connection processing on the enhanced features to obtain the output feature map of the current layer. S2 is used to extract hierarchical feature representations from local to global layers. For example... Figure 2 As shown:

[0099] The main branch maintains the original SASF single-scale modeling;

[0100] The auxiliary branch uses a three-way parallel dilated convolution (d=1,3,5).

[0101] The effective receptive field is 3×3→7×7→11×11;

[0102] The dynamic weight calculation process is global average pooling → fully connected → Softmax;

[0103] Feature fusion is achieved by weighted summation based on weights α1:α3:α5;

[0104] Gated fusion is achieved by dynamically adjusting the weights of the main and auxiliary branches using σ(G(z)).

[0105] In one embodiment, the main branch is used to perform the following steps:

[0106] The input two-dimensional initial feature map is scanned in four directions by a four-directional scanner and converted into a one-dimensional sequence.

[0107] One-dimensional sequences are input into a selective state-space model for processing to capture long-range dependencies;

[0108] The output of the selective state-space model is remapped back to two-dimensional space through an expanded convolutional weight reconstructor.

[0109] This step addresses spatial distortion through a scanning strategy and enhances local structure perception using dilated convolution.

[0110] The main branch execution process is as follows:

[0111] ① The input features are sequentially processed through the Structure-Aware State Fusion (SASF) module for spatial dependency modeling, with a computational complexity of O(HW):

[0112]

[0113] in It is the current state vector, storing the hidden states for sequence modeling. It is the state vector from the previous moment, conveying historical information. It is a content-dependent state transition matrix (generated through linear projection), which controls the state evolution path. It is the input projection matrix, which maps the input features to the state space. It is the current input feature vector, that is, the t-th element in the sequence X_seq output by the four-way scanner from the input feature map of the current layer MS-SMamba block. It is the Hadamard product (element-wise product), which enables non-linear feature interaction.

[0114] In this embodiment, the scanning step size of the four-way scanner along the H / W direction can be 4 pixels.

[0115] It should be noted that the four-way scanning performs prefix accumulation and local updates only along the row and column directions of the input feature map, which can be directly mapped to a one-dimensional convolution / reduction operator. This avoids the random indexing and large-scale matrix multiplication operations found in global attention methods, and possesses characteristics such as continuous memory, simple computation kernel, and high parallelism. Furthermore, each pixel only performs a state update with its adjacent positions once, resulting in a single-direction computational complexity of O(HW) and a constant multiple of O(HW) for all four directions. This ensures that the complexity of a single forward inference increases linearly, significantly reducing the quadratic complexity compared to self-attention's O((HW)²).

[0116] In one embodiment, the step of the auxiliary branch performing multi-scale feature extraction on the first feature includes:

[0117] Multiple dilation convolution operations with different dilation rates are performed in parallel on the first feature to extract features at different scales.

[0118] In this embodiment, the steps of the auxiliary branch generating content-adaptive dynamic weights based on the global statistical information of the first feature, performing weighted fusion of multi-scale features, and outputting the second feature include:

[0119] The global context feature vector of the first feature is extracted by global average pooling;

[0120] The global context feature vector is mapped to the same dimension as the number of branches in the dilated convolution through a fully connected layer.

[0121] Normalization is performed using the Softmax function to generate dynamic weight coefficients for each dilated convolution branch.

[0122] The second feature is obtained by weighting and fusing the corresponding multi-scale spatial features using dynamic weighting coefficients.

[0123] In one embodiment, the steps of adaptive weighted fusion using a fusion-gated network include:

[0124] The first feature and the second feature are concatenated along the channel dimension to generate a joint feature;

[0125] The joint features are input into the convolutional bottleneck network, and an activation function is applied to the output of the bottleneck network to generate a pixel-level gated weight map γ∈[0,1].

[0126] Based on the gating weight map, the enhanced feature h_final = γ⊙h_main + (1-γ)⊙h_fused is obtained and fused, where h_main is the first feature, h_fused is the second feature, and ⊙ is the Hadamard product.

[0127] This step implements the contribution weights of dynamic equilibrium sequence modeling and spatial perception.

[0128] In one embodiment, the operation of global feature aggregation of the output feature map of the last layer MS-SMamba block in S3 includes processing the output feature map through a global average pooling layer and then inputting it into a fully connected layer to output the category probability of the medical image.

[0129] The specific execution process of the auxiliary branch is as follows:

[0130] like Figure 3 As shown, the three parallel branches are as follows:

[0131] Branch-d=1: Standard 3×3 depthwise separable convolution (k=3×3, d=1), output feature h^{(1)}∈R^{B×C×H×W}, effective receptive field 3×3 pixels, capturing local texture details;

[0132] Branch-d=3: 3×3 depthwise separable convolution with dilation rate of 3 (k=3×3, d=3), output feature h^{(3)}∈R^{B×C×H×W}, effective receptive field 7×7 pixels, modeling organ-level anatomical structures;

[0133] Branch-d=5: 3×3 depthwise separable convolution with dilation rate of 5 (k=3×3, d=5), output feature h^{(5)}∈R^{B×C×H×W}, effective receptive field of 11×11 pixels, capturing long-range dependencies between organs.

[0134] The effective receptive field (RF^{(1)}=3,RF^{(3)}=7,RF^{(5)}=11) is controlled by the expansion rate, covering the spatial range from local to global.

[0135] Through depthwise separable optimization, convolution operations are performed independently on each input channel, reducing the number of parameters (from C²k² to Ck²). Multi-channel features are fused using 1×1 convolutions to restore channel dimensions. Standard convolution complexity O(9HWC) vs. depthwise separable convolution O(3HWC), a 3x speedup. This reduces computational complexity from O(C²k²HW) to O(Ck²HW) while maintaining linear computational characteristics.

[0136] The Cross-Scale Interaction (CSI) module extracts multi-scale features in parallel and fuses them with the main branch:

[0137]

[0138]

[0139] in is the scale weight coefficient (scalar), reflecting the importance of features at the d-th scale; W is the learnable weight matrix, generating multi-scale weights; b is the bias vector, enhancing the model's expressive power; and Softmax is a normalized exponential function. It is a global average pooling output, which compresses the spatial dimensions to generate statistical descriptors. These are convolutional features with an inflation rate of d, where d∈{1,3,5} corresponds to local / organ / global scales. It is the result of dynamic weighted summation, which realizes content-aware multi-scale feature integration, enabling the model to automatically adjust different scales according to the feature distribution of the input image.

[0140] This step enables multi-scale feature extraction and content adaptive fusion.

[0141] like Figure 4 As shown, the dynamic weighted summation mechanism enables content-adaptive feature fusion:

[0142] Global average pooling g = AvgPool2D(x) compresses the spatial dimension to 1×C, generating a global feature vector g∈R^C. A parameterized fully connected layer f_θ(g) = Wg + b (W∈R^{3×C}, b∈R^3) maps g to a 3C dimension (W∈R³×C). Softmax normalization generates probability weights α_d = Softmax(f_θ(g))_d, ensuring ∑α_d = 1, satisfying the probability distribution constraint. α_d∈R^3, and the weights α_d reflect the importance of features at different scales, with α1 biased towards local details and α5 biased towards the global context. Multi-scale features are weighted and summed according to α1:α3:α5 to generate a content-adaptive multi-scale fused feature h_fused = α1·h^{(1)} + α3·h^{(3)} + α5·h^{(5)}. Error is backpropagated using a chain rule to optimize W, b, and the main branch parameters.

[0143] Softmax function:

[0144]

[0145] in is the i-th element of the output vector, representing the predicted probability of the i-th class, with a value range of [0,1]. z is the input vector (logits) with a dimension of K×1 (K is the total number of classes). It is the i-th element of the input vector z (the unnormalized class score). Yes The result of the exponentiation operation is guaranteed to be positive. It is the sum of all category index values ​​(normalized denominator to ensure the probability sum is 1), and K is the total number of categories.

[0146] Global average pooling output :

[0147]

[0148] in H is the input feature map (from the output of the state space modeling branch), H is the feature map height, and W is the feature map width. is the feature vector at spatial location (i,j), and g is the global average pooling output.

[0149] In one embodiment, the residual connection unit and the feedforward network ensure stable gradient propagation through residual connections, and the feedforward network (MLP Block) enhances the representation of nonlinear features.

[0150]

[0151] in It is the module input feature (dimension B×(H / 4)×(W / 4)×D). It is the core computation of the MS-SMamba block (including sub-modules such as SASF, CSI, and dynamic gating). These are the output features after the residual connection. Their function is to ensure stable gradient propagation and alleviate the gradient vanishing problem during deep network training.

[0152] S3: Perform global feature aggregation on the output feature map of the last layer MS-SMamba block to obtain a global feature vector; perform classification based on the global feature vector to output the classification result of the medical image.

[0153] The specific execution process of S3 is as follows:

[0154] ① Flatten the 3D feature map (B×H / 32×W / 32×D) into a 2D matrix (B×(H / 32×W / 32)×D), and after flattening, transpose it after layer normalization to (B×D×(H / 32×W / 32)). Perform one-dimensional adaptive global average pooling along the spatial dimension to generate a compact global feature vector (B×D).

[0155] ② Classification based on the fusion of main and auxiliary branch features (B×2D→B×D) using dynamic gating networks:

[0156]

[0157]

[0158]

[0159] Where z is the concatenation feature, Concat represents concatenation, and h main It is the main branch output, carrying global long-range dependency information, h aux This is the auxiliary branch output, carrying multi-scale spatial awareness features derived from state space modeling. `dim` is the dimension identifier; channel dimensions are concatenated to maintain spatial structural integrity. `z=Concat([h...` main ,h aux ],dim=1) is equivalent to channel-wise concatenation, γ is the gate weight (scalar) in the range [0,1], output by the Sigmoid function, σ is the Sigmoid activation function from multi-scale convolution, ReLU is the ReLU activation function, BN is batch normalization to accelerate training convergence, Conv1×1 is a 1×1 convolution to achieve channel dimensionality reduction / upgrading, h final It is a dynamic gating network fusion output.

[0160] Sigmoid activation function:

[0161]

[0162] Where x is the input value (from the gated convolution output) and the unnormalized gate weights. This is the output value, which compresses the input to the [0,1] interval and represents the probability weights. Its main function is to generate pixel-level gating weights γ∈[0,1] to achieve adaptive fusion of the main and auxiliary branches: γ≈1: dependent on the main branch (Mamba sequence modeling capability); γ≈0: dependent on the auxiliary branch (multi-scale spatial perception capability).

[0163] ReLU activation function:

[0164]

[0165] Where x is the input feature value (from the output of the BN layer), the original feature to be activated, and max(0,x) is the output feature value, which retains positive features and suppresses negative features (enhancing nonlinear expressive power).

[0166] BN (Batch Normalization):

[0167]

[0168] Where x i It is the i-th element of the input features, the original feature to be normalized, m is the batch size, and μ is the number of samples in the current batch. B It is the feature mean of the current batch, a centered feature distribution. It represents the feature variance of the current batch, scaling the feature distribution range. It is a numerically stable term to prevent the denominator from being zero. These are the normalized features, scaled to zero mean and unit variance. γ and β are scaling and shifting factors, restoring the feature's expressive power (avoiding information loss). i It is the output of the BN layer, which stabilizes the feature distribution and accelerates training convergence.

[0169] 1×1 convolution (Conv1×1):

[0170]

[0171] in It is the input feature map, with input channel c at position (i,j). in The feature values, the original features to be processed. These are the convolution kernel weights, and the output channel c. out With input channel c in Connection weights between features, linear transformations between feature channels, It's the bias term, output channel c out The bias enhances the model's expressive power. It is the output feature map, and the output channel c is located at position (i,j). out The eigenvalues, the features after channel dimension transformation, c in The number of input channels determines the dimension of the weight matrix, c. out The number of output channels determines the dimension of the transformed features.

[0172] ③ The output feature map is processed by a global average pooling layer and a linear classification layer to output the final predicted probability (B×C).

[0173] The following is an example of the overall image classification processing logic of an embodiment of the present invention (taking a 224×224 resolution as an example):

[0174] ① Input preprocessing: Divide the 224×224×3 medical image into 56×56 4×4 image blocks;

[0175] ② Feature embedding: Initial features are generated through a 768×(4×4×3) linear projection layer;

[0176] ③ Feature extraction: Processed through multiple MS-SMamba blocks, which can be 12 to 24 layers;

[0177] ④ Feature aggregation: Global average pooling followed by dynamic gating fusion;

[0178] ⑤ Classification output: The fully connected layer outputs the classification probability.

[0179] The following describes the specific implementation of MS-SMamba in medical image classification tasks:

[0180] (1) Data preprocessing

[0181] Image normalization: Normalize the input image to the range of [0,1] to eliminate differences in device acquisition.

[0182] Data augmentation: Generalization ability is enhanced by random rotation (±15°), elastic deformation (σ=15), and noise injection (Gaussian noise σ=0.1).

[0183] (2) Model Configuration

[0184] Input size: 224×224×3 (supports multi-scale input, such as 384×384×3).

[0185] Hidden dimension: D=768, the number of channels gradually increases with network depth (256→512→768).

[0186] Classification Header: Global average pooling followed by a fully connected layer, outputting multi-class probabilities.

[0187] (3) Training strategy

[0188] Optimizer: AdamW optimizer, initial learning rate 1e-4, weight decay 1e-2.

[0189] Loss function: Cross-entropy loss function combined with L2 regularization (λ=1e-4).

[0190] Batch size: 128, gradient accumulation steps: 2 (for memory-constrained scenarios).

[0191] The following visual verification demonstrates the multi-scale modeling capability:

[0192] Tools: Grad-CAM++ was used to generate a heat map of the lesion area.

[0193] Results: MS-SMamba showed improved sensitivity to microcalcifications and improved accuracy in locating tumor margins.

[0194] like Figure 5 As shown, the comparison results of the feature heatmaps are as follows:

[0195] CNN (ResNet50) comparison: Traditional CNNs only activate local regions and cannot connect structures between organs.

[0196] ViT (Swin) comparison: ViT loses details due to patching, and has a weak response to small lesions.

[0197] MS-SMamba: The main branch captures global dependencies, while the auxiliary branches enhance local details, and the heatmap is highly consistent with the clinical gold standard.

[0198] The second aspect of this invention also discloses a medical image classification system based on multi-scale spatial state modeling, according to the first aspect of the invention, comprising:

[0199] The input preprocessing and feature embedding module receives the input medical image and divides it into multiple non-overlapping image patches; it then maps each image patch to a feature space of a preset dimension through a learnable linear projection layer to obtain an initial feature map.

[0200] Multi-layered stacked MS-SMamba block modules are used for layer-by-layer feature extraction from the initial feature map. Each MS-SMamba block includes:

[0201] The main branch is used to perform selective state-space modeling scanning through the state-space model, capture long-range dependencies, and output the first feature.

[0202] The auxiliary branch is used to extract multi-scale features from the first feature and generate content-adaptive dynamic weights based on the global statistical information of the first feature. It then performs weighted fusion of the multi-scale features and outputs the second feature.

[0203] A dynamic gated fusion network is used to generate a gated weight map based on a first feature and a second feature, and then adaptively weightedly fuses the first feature and the second feature based on the gated weight map to obtain the enhanced feature.

[0204] Residual connection units and feedforward networks are used to perform residual connection processing on the enhanced features to obtain the output feature map of the current layer;

[0205] The global feature aggregation and classification module is used to perform global feature aggregation on the output feature map of the last layer MS-SMamba block to obtain a global feature vector; and to classify and output the classification result of the medical image based on the global feature vector.

[0206] The second aspect of the present invention is used to perform all the steps in the first aspect of the present invention.

[0207] This invention achieves the following technological breakthroughs in medical image classification tasks through innovative design of multi-scale spatial state space modeling and cross-scale interaction mechanisms:

[0208] ① Diagnostic accuracy: 3%-5% improvement in accuracy on datasets such as Kvasir and PAD-UFES-20;

[0209] ② Computational efficiency: 29M parameters, inference latency of 5.8ms (batch processing 128), suitable for embedded devices;

[0210] ③ Clinical interpretability: Saliency maps and heatmaps validate the model by focusing on key diagnostic regions.

[0211] This invention provides an efficient, reliable, and interpretable solution for the field of medical AI, with significant potential for clinical translation.

[0212] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0213] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A medical image classification method based on multi-scale spatial state modeling, characterized in that, Includes the following steps: S1: Divide the input medical image into multiple non-overlapping image patches; map each image patch to a feature space of a preset dimension through a learnable linear projection layer to obtain an initial feature map; S2: The initial feature map is input into a multi-layered stacked MS-SMamba block for layer-by-layer feature extraction; wherein each MS-SMamba block includes: The main branch is used to perform selective state-space modeling scanning through the state-space model, capture long-range dependencies, and output the first feature. An auxiliary branch is used to extract multi-scale features from the first feature, generate content-adaptive dynamic weights based on the global statistical information of the first feature, perform weighted fusion of the multi-scale features, and output the second feature. A dynamic gated fusion network is used to generate a gated weight map based on the first feature and the second feature, and to adaptively weight and fuse the first feature and the second feature based on the gated weight map to obtain enhanced features; Residual connection units and feedforward networks are used to perform residual connection processing on the enhanced features to obtain the output feature map of the current layer; S3: Perform global feature aggregation on the output feature map of the last layer MS-SMamba block to obtain a global feature vector; classify and output the classification result of the medical image based on the global feature vector.

2. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, In step S1, after mapping each image patch to a feature space of a preset dimension through a learnable linear projection layer to obtain the feature embedding vector, the following steps are also included: By using depthwise separable convolution kernels, local spatial relationships are perceived through convolution based on the feature embedding vectors, and positional codes are dynamically generated and fused, so that the output initial feature map includes spatial positional information.

3. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, The main branch is used to perform the following steps: The input two-dimensional initial feature map is scanned in four directions by a four-way scanner and converted into a one-dimensional sequence. The one-dimensional sequence is input into a selective state-space model for processing to capture long-range dependencies; The output of the selective state-space model is remapped back to two-dimensional space through an expanded convolutional weight reconstructor.

4. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, The auxiliary branch performs multi-scale feature extraction on the first feature, including the following steps: Multiple dilation convolution operations with different dilation rates are performed in parallel on the first feature to extract features at different scales.

5. A medical image classification method based on multi-scale spatial state modeling according to claim 4, characterized in that, The steps of generating content-adaptive dynamic weights based on the global statistical information of the first feature, weighted fusing of multi-scale features, and outputting the second feature include: The global context feature vector of the first feature is extracted by global average pooling; The global context feature vector is mapped to the same dimension as the number of branches in the dilated convolution through a fully connected layer. Normalization is performed using the Softmax function to generate dynamic weight coefficients for each dilated convolution branch. The corresponding multi-scale spatial features are weighted and fused using the dynamic weighting coefficients to obtain the second feature.

6. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, The steps for adaptive weighted fusion in the fusion gating network include: The first feature and the second feature are concatenated along the channel dimension to generate a joint feature; The joint features are input into a convolutional bottleneck network, and an activation function is applied to the output of the bottleneck network to generate a pixel-level gated weight map γ∈[0,1]. Based on the gating weight map, the enhanced feature h_final = γ⊙h_main + (1-γ)⊙h_fused is fused, where h_main is the first feature, h_fused is the second feature, and ⊙ is the Hadamard product.

7. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, The operation of global feature aggregation of the output feature map of the last layer MS-SMamba block in S3 includes processing the output feature map by a global average pooling layer and then inputting it into a fully connected layer to output the category probability of the medical image.

8. A medical image classification system based on multi-scale spatial state modeling, according to any one of claims 1-7, characterized in that, include: The input preprocessing and feature embedding module is used to receive the input medical image and divide it into multiple non-overlapping image patches; An initial feature map is obtained by mapping each image patch to a feature space of a preset dimension through a learnable linear projection layer; A multi-layered stacked MS-SMamba block module is used to perform layer-by-layer feature extraction on the initial feature map. Each MS-SMamba block includes: The main branch is used to perform selective state-space modeling scanning through the state-space model, capture long-range dependencies, and output the first feature. An auxiliary branch is used to extract multi-scale features from the first feature, generate content-adaptive dynamic weights based on the global statistical information of the first feature, perform weighted fusion of the multi-scale features, and output the second feature. A dynamic gated fusion network is used to generate a gated weight map based on the first feature and the second feature, and to adaptively weight and fuse the first feature and the second feature based on the gated weight map to obtain enhanced features; Residual connection units and feedforward networks are used to perform residual connection processing on the enhanced features to obtain the output feature map of the current layer; The global feature aggregation and classification module is used to perform global feature aggregation on the output feature map of the last layer MS-SMamba block to obtain a global feature vector; and to perform classification based on the global feature vector to output the classification result of the medical image.

Images