Medical image classification method and system based on multi-scale spatial state modeling

Abstract

The application discloses a medical image classification method and system based on multi-scale space state modeling, which comprises the following steps: firstly, dividing the input medical image into multiple non-overlapping image blocks, and mapping to the feature space through a learnable linear projection layer to obtain an initial feature map; then, using a multi-layer stacked MS-SMamba block to extract features layer by layer, wherein each MS-SMamba block comprises a main branch, an auxiliary branch, a dynamic gating fusion network, a residual connection unit and a feedforward network, realizing long-range dependence relationship capture and multi-scale feature fusion; finally, processing the output feature map of the last layer through a global feature aggregation and classification module, generating a global feature vector and outputting a classification result. The application improves the capture ability of complex pathological features in medical images, improves the calculation efficiency and clinical applicability, and is suitable for disease screening, auxiliary decision-making and other scenes in medical image diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image analysis, in particular to a medical image classification method and system based on multi-scale space state modeling. BACKGROUND

[0002] As the core link of disease diagnosis, medical image classification is undergoing a transformation from manual reading to intelligent analysis. With the improvement of medical imaging device resolution (such as 3D ultrasound and high-resolution CT) and the popularization of multi-modal imaging technology, medical image data presents the following characteristics:

[0003] I. Data complexity:

[0004] Lesion heterogeneity: the same disease shows significant morphological differences at different stages (such as the morphological evolution of tumors from small calcification to invasive growth), and similar image features often exist between different diseases (such as the overlap of ground glass shadows between pneumonia and tuberculosis).

[0005] Structural hierarchy: the human anatomy presents a multi-level relationship of organ-tissue-cell (such as the anatomical coupling of liver tumor and intrahepatic blood vessels), and lesions often exhibit complex characteristics such as small targets (diameter <2mm) and weak boundaries (such as early liver fibrosis).

[0006] Modality diversity: a single modality cannot fully represent pathological information (such as MRI providing soft tissue contrast and PET reflecting metabolic activity), and multi-modal fusion is the key to improving diagnostic efficiency.

[0007] II. Structural particularity:

[0008] Diagnosis efficiency bottleneck: WHO reports show that global medical imaging data is growing at an annual rate of 30%.

[0009] Diagnosis accuracy requirement: taking breast cancer screening as an example, the US FDA requires the false negative rate of screening systems to be less than 1%, while the sensitivity of traditional CAD systems in dense breasts is less than 70%.

[0010] Real-time constraints: Intraoperative navigation and other clinical scenarios require model inference delay to be less than 10ms, and existing methods cannot meet the stringent real-time requirements.

[0011] Clinical demand urgency: WHO reports show that global medical imaging data is growing at an annual rate of 30%, and there is an urgent need for efficient and accurate automated classification tools.

[0012] Current mainstream medical image classification techniques can be divided into three categories: solutions based on convolutional neural networks (CNN), visual Transformers (ViT), and state space models (SSM), but all have significant limitations.

[0013] With the development of deep learning technology, Convolutional Neural Network (CNN) and Vision Transformer (ViT) have become the mainstream methods, but there are still significant limitations in medical image analysis:

[0014] I. Based on the convolutional neural network (CNN) scheme, multi-scale features are extracted through parallel multi-dilation convolution branches, multi-scale context features are captured through different sampling rates of dilated convolution, and the detail recovery capability is enhanced through feature map splicing. However, the parallel convolution branch with fixed dilation rate cannot adapt to the change of lesion size, the traditional jump connection has feature alignment error, and the multi-scale feature fusion lacks content adaptive weight distribution, and the parameter quantity of 3D CNN increases cubically with the expansion of dimension. Therefore, although the parallel dilated convolution branch can capture multi-scale features, the fixed receptive field design leads to insufficient sensitivity for small target detection, and the model parameter quantity increases exponentially with the increase of depth.

[0015] II. Based on the vision Transformer (ViT) scheme, global dependency modeling is achieved through block self-attention, mobile window mechanism is introduced to balance computational efficiency and global perception, and contrast learning strategy is adopted to improve cross-center generalization ability. However, patch division leads to the breakage of key anatomical structures, and position encoding is difficult to adapt to the complex topological relationship of medical images, the long sequence modeling of ViT leads to a sharp increase in memory occupation, making it difficult to deploy on embedded devices, and it lacks robustness to unstructured noise such as speckle noise of ultrasound images and artifacts of MRI. Therefore, the global self-attention mechanism of ViT performs well in natural images, but the small target characteristics of medical images lead to sparse attention weight distribution, resulting in a decrease in actual computational efficiency.

[0016] III. Based on the state space model (SSM) scheme, Mamba architecture is first introduced into medical image segmentation task, efficient 3D medical image segmentation is achieved through variant state space modeling, and window attention and state space scanning strategies are fused. However, existing SSM scanning strategies (such as four-way scanning) artificially distort spatial adjacency relationships, and existing SSM models rely on fixed scanning steps and cannot adaptively adjust the perception range, resulting in inference delay in 3D medical image segmentation, making it difficult to meet the real-time requirements of intraoperative navigation. Therefore, although its performance is improved through state dimension expansion, the computational complexity of 3D convolution leads to an increase in inference delay, making it difficult to meet the clinical real-time requirements.

[0017] Therefore, how to provide a medical image classification method and system that can balance multi-scale feature extraction, spatial structure preservation, and computational efficiency optimization is a problem that needs to be solved by those skilled in the art. SUMMARY

[0018] Therefore, the application provides a medical image classification method and system based on multi-scale space state modeling.

[0019] To achieve the above object, the application adopts the following technical scheme:

[0020] The application first proposes a medical image classification method based on multi-scale space state modeling, including the following steps:

[0021] S1: divide the input medical image into multiple non-overlapping image blocks; map each image block to a feature space of a preset dimension through a learnable linear projection layer to obtain an initial feature map;

[0022] S2: input the initial feature map into a multi-layer stacked MS-SMamba block for layer-by-layer feature extraction; each MS-SMamba block includes:

[0023] a main branch for selective state space modeling scanning through a state space model to capture long-range dependencies and output a first feature;

[0024] an auxiliary branch for multi-scale feature extraction on the first feature and generation of content-adaptive dynamic weights according to global statistical information of the first feature, weighted fusion of the multi-scale features, and output of a second feature;

[0025] a dynamic gating fusion network for generating a gating weight map according to the first feature and the second feature, adaptive weighted fusion of the first feature and the second feature based on the gating weight map, and obtaining an enhanced feature;

[0026] a residual connection unit and a feedforward network for residual connection processing of the enhanced feature to obtain an output feature map of the current layer;

[0027] S3: globally aggregate the output feature map output by the last MS-SMamba block to obtain a global feature vector; and output a classification result of the medical image according to the global feature vector.

[0028] Preferably, after the feature embedding vector is obtained by mapping each image block to a feature space of a preset dimension through a learnable linear projection layer in S1, the following steps are further included:

[0029] a deep separable convolution kernel is used to convolve and perceive local spatial relationships according to the feature embedding vector, dynamically generate position encoding, and fuse them, so that the initial feature map output includes spatial position information.

[0030] Preferably, the main branch is configured to perform the following steps:

[0031] The input two-dimensional initial feature map is scanned along four directions by a four-way scanner to convert it into a one-dimensional sequence;

[0032] The one-dimensional sequence is input into a selective state space model for processing to capture long-range dependencies;

[0033] The output of the selective state space model is remapped back to a two-dimensional space by a dilated convolution weight re-mapper.

[0034] Preferably, the auxiliary branch performs the step of multi-scale feature extraction on the first feature, which includes:

[0035] Parallelly performing dilated convolution operations with different dilation rates on the first feature to extract features of different scales, respectively.

[0036] Preferably, the auxiliary branch generates content-adaptive dynamic weights according to the global statistical information of the first feature, and performs weighted fusion on the multi-scale features to output the second feature, which includes:

[0037] Extracting a global context feature vector of the first feature by global average pooling;

[0038] Mapping the global context feature vector to a dimension same as the number of dilated convolution branches by a fully connected layer;

[0039] Normalizing by a Softmax function to generate dynamic weight coefficients corresponding to each dilated convolution branch;

[0040] Using the dynamic weight coefficients to perform weighted fusion on the corresponding multi-scale spatial features to obtain the second feature.

[0041] Preferably, the step of adaptive weighted fusion performed by the fusion gating network includes:

[0042] Concatenating the first feature and the second feature in the channel dimension to generate a joint feature;

[0043] Inputting the joint feature into a convolutional bottleneck network, and using an activation function on the output of the bottleneck network to generate a pixel-level gating weight map γ∈[0,1];

[0044] According to the gating weight map, an enhanced feature h_final=γ⊙h_main+(1-γ)⊙h_fused is obtained for fusion, where h_main is the first feature, h_fused is the second feature, and ⊙ is the Hadamard product.

[0045] Preferably, the operation of globally aggregating the output feature map output by the last MS-SMamba block in S3 includes inputting the output feature map into a fully connected layer after processing by a global average pooling layer, and outputting a class probability of the medical image.

[0046] The application further provides a medical image classification system based on the medical image classification method based on multi-scale spatial state modeling.

[0047] The input preprocessing and feature embedding module is configured to receive an input medical image and divide the medical image into a plurality of non-overlapping image blocks, and map each image block to a preset dimension feature space through a learnable linear projection layer to obtain an initial feature map.

[0048] The multi-layer stacked MS-SMamba block module is configured to perform layer-by-layer feature extraction on the initial feature map, and each MS-SMamba block includes:

[0049] The main branch is configured to perform selective state space modeling scanning through a state space model to capture long-range dependencies and output a first feature.

[0050] The auxiliary branch is configured to perform multi-scale feature extraction on the first feature, generate content-adaptive dynamic weights according to global statistical information of the first feature, and perform weighted fusion on the multi-scale features to output a second feature.

[0051] The dynamic gating fusion network is configured to generate a gating weight map according to the first feature and the second feature, and perform adaptive weighted fusion on the first feature and the second feature based on the gating weight map to obtain an enhanced feature.

[0052] The residual connection unit and the feedforward network are configured to perform residual connection processing on the enhanced feature to obtain an output feature map of a current layer.

[0053] The global feature aggregation and classification module is configured to globally aggregate the output feature map output by the last MS-SMamba block to obtain a global feature vector, and output a classification result of the medical image according to the global feature vector.

[0054] Compared with the prior art, the medical image classification method and system based on multi-scale spatial state modeling provided by the application have the following beneficial effects through the innovative space-scale joint modeling mechanism and dynamic fusion architecture:

[0055] 1. Precise capture of multi-scale pathological features. By introducing a multi-scale structure perception module and a cross-scale dilated convolution array, the model can adaptively match pathological features of different scales.

[0056] ① The detection sensitivity of micro lesions with a diameter of less than 2 mm is improved;

[0057] ② The modeling integrity of complex organ structures is improved;

[0058] ③ Multi-scale feature fusion effectively suppresses intra-class differences.

[0059] 2. Spatial-sequence modeling efficiency breakthrough. The dynamic gating fusion mechanism and hardware-friendly scanning strategy are created to realize dual optimization of computational efficiency and representation ability:

[0060] ① The single forward calculation complexity remains O(HW) linear growth, which is reduced by 3 orders of magnitude compared to ViT;

[0061] ② The inference throughput of 1540 img / s is achieved on NVIDIA 4090D GPU, which is significantly faster than 2D Mamba;

[0062] ③ The parameter amount is compressed to 29M, which is significantly reduced compared to traditional ViT, meeting the deployment requirements of embedded devices.

[0063] 3. Significant enhancement of medical scene adaptability. Through medical explainability enhancement design and noise robustness optimization, the clinical applicability is greatly improved:

[0064] ① The significant map analysis shows that the attention of the lesion area is improved;

[0065] ② The tolerance to speckle noise of ultrasound images is improved;

[0066] ③ Supports multiple resolutions from 224x224 to 512x512, with constant parameter amount and linear growth of FLOPs.

[0067] 4. Long-range dependency modeling capability innovation. Based on state space scanning strategy optimization and content adaptive weight allocation, the spatio-temporal modeling limitations of traditional models are broken through:

[0068] ① Global dependency capture based on cross-scale interaction module, accurate modeling of inter-organ anatomical structure correlation;

[0069] ② The context awareness range of small targets is expanded, which is effectively improved compared to the fixed scanning strategy;

[0070] ③ The long sequence modeling memory occupancy is reduced through depthwise separable convolution optimization.

[0071] 5. Overall improvement of clinical deployment feasibility. Through lightweight gating mechanism and adaptive computing scheduling, the needs of severe medical scenarios are met:

[0072] ① The inference delay is controlled at 5.8ms, with a batch size of 128, meeting the real-time diagnosis requirements;

[0073] ②Mobile deployment power consumption is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0075] Figure 1 The overall architecture diagram of the medical image classification method model based on multi-scale space state modeling provided by the embodiments of the present application is shown in the figure.

[0076] Figure 2 The principle diagram of the MS-SMamba block in the auxiliary branch provided by the embodiments of the present application is shown in the figure.

[0077] Figure 3 The principle diagram of the cross-scale interaction module based on multi-scale dilated convolution provided by the embodiments of the present application is shown in the figure.

[0078] Figure 4 The principle diagram of the dynamic weight mechanism provided by the embodiments of the present application is shown in the figure.

[0079] Figure 5 The feature heat map comparison result provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0080] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0081] The first aspect of the embodiments of the present application discloses a medical image classification method based on multi-scale space state modeling, comprising the following steps:

[0082] S1: dividing the input medical image into a plurality of non-overlapping image blocks; mapping each image block to a feature space of a preset dimension through a learnable linear projection layer to obtain an initial feature map. S1 is used to complete the conversion of image to sequence feature and establish a spatial position-aware semantic representation.

[0083] In one embodiment, after the feature embedding vector is obtained by mapping each image block to a feature space of a preset dimension through a learnable linear projection layer in S1, the following steps are further included:

[0084] With the depth separable convolution kernel, the local spatial relationship is perceived according to the feature embedding vector, the position encoding is dynamically generated and fused, and the spatial position information is included in the output initial feature map.

[0085] As shown in Figure 1 , the input image HxWxC is converted into (H / 4)x(W / 4)x16C feature tensor by using 4x4 non-overlapping block strategy. It is mapped to D-dimensional feature space through a learnable linear projection layer.

[0086] The specific implementation process of S1 is as follows:

[0087] ①The input medical image (BxHxWxC) is first divided into 4x4 pixel non-overlapping image blocks (Bx(H / 4)x(W / 4)x16C) by the patch division unit, and each block is flattened into a 16C-dimensional feature vector, where B is the batch size (BatchSize), indicating the number of image samples processed in a single forward calculation, and the calculation parallelism and efficiency are improved through batch processing during deep learning training / inference, H is the height (pixel unit) of the input medical image, W is the width (pixel unit) of the input medical image, and C is the channel depth of the input image (Channel Depth), indicating the color or modal dimension of the image.

[0088] ②The linear projection layer maps each image block to a D-dimensional feature space (Bx(H / 4)x(W / 4)xD):

[0089]

[0090] Wherein is the flattened feature tensor (4x4 non-overlapping block) output by the image patch division unit, is a learnable projection matrix that maps 16C-dimensional features to D-dimensional hidden space, D is the hidden dimension of the feature (Hidden Dimension), i.e. the number of channels of the feature inside the model, which defines the representation ability of the model hidden layer, the larger the value, the stronger the representation ability (but the higher the calculation overhead), is the feature tensor after projection. Its role is to convert the original pixel block into high-dimensional semantic features to provide a basis for subsequent sequence modeling.

[0091] ③The CPE module dynamically generates position encoding (Bx(H / 4)x(W / 4)xD) according to the input feature:

[0092]

[0093] Wherein Z is the feature tensor after linear projection, is a depth separable convolution kernel (kernel size kxk, usually k=3), is a dynamically generated position encoding. The role is to perceive local spatial relationships through convolution, inject position sensitivity to features, and solve the problem of spatial structure distortion caused by image block serialization.

[0094] S2: input the initial feature map into the multi-layer stacked MS-SMamba block for layer-by-layer feature extraction; wherein each layer of the MS-SMamba block comprises:

[0095] a main branch for selective state space modeling scanning by a state space model to capture long-range dependencies and output first features;

[0096] an auxiliary branch for multi-scale feature extraction on the first features and generating content-adaptive dynamic weights according to global statistical information of the first features, and weighted fusion of the multi-scale features to output second features;

[0097] a dynamic gating fusion network for generating a gating weight map according to the first features and the second features, and adaptively weighting and fusing the first features and the second features based on the gating weight map to obtain enhanced features;

[0098] a residual connection unit and a feedforward network for residual connection processing on the enhanced features to obtain output feature maps of the current layer. S2 is used for layer-by-layer extraction of hierarchical feature representation from local to global. As shown in Figure 2

[0099] the main branch maintains single-scale modeling of the original SASF;

[0100] the auxiliary branch uses three-way parallel dilated convolution (d=1, 3, 5);

[0101] the effective receptive field is 3x3→7x7→11x11;

[0102] the dynamic weight calculation process is global average pooling→full connection→Softmax;

[0103] weighted sum with weights α1:α3:α5 to realize feature fusion;

[0104] gate fusion is realized by dynamically adjusting the main / auxiliary branch weights through σ(G(z)).

[0105] In one embodiment, the main branch is used to perform the following steps:

[0106] scan the input two-dimensional initial feature map in four directions through a four-way scanner to convert it into a one-dimensional sequence;

[0107] input the one-dimensional sequence into a selective state space model for processing to capture long-range dependencies;

[0108] ​The output of the selective state space model is remapped back to a two-dimensional space by an extended convolution weight re-mapper.

[0109] This step solves the spatial distortion problem by scanning strategy, and uses extended convolution to enhance local structure perception.

[0110] The specific implementation process of the main branch is as follows:

[0111] ①The input features are sequentially input into the structure perception state fusion module (SASF) for spatial dependence modeling, and the computational complexity is O(HW):

[0112]

[0113] wherein is the current state vector, which stores the hidden state of sequence modeling, is the state vector of the previous moment, which transmits historical information, is the content-dependent state transition matrix (generated by linear projection), which is generated by linear projection, and controls the state evolution path, is the input projection matrix, which maps the input features to the state space, is the current input feature vector, which is the t-th element in the sequence X_seq output by the four-way scanner for the input feature map of the current layer MS-SMamba block, and for is the Hadamard product (element-wise multiplication), which realizes nonlinear feature interaction.

[0114] In this embodiment, the scanning step of the four-way scanner along the H / W direction can be 4 pixels.

[0115] It should be noted that the four-way scanning only performs prefix accumulation and local update along the row and column directions of the input feature map, which can be directly mapped to a one-dimensional convolution / reduction operator, avoiding random indexing and large-scale matrix multiplication in global attention methods, and having the characteristics of continuous memory, simple calculation kernel, high parallelism, etc. At the same time, each pixel point only updates the state with the adjacent position once, and the single-direction computational complexity is O(HW), and the four directions are constant times O(HW), which ensures that the single forward reasoning complexity grows linearly, which is significantly reduced compared with the quadratic complexity of self-attention O((HW)²).

[0116] In one embodiment, the step of the auxiliary branch performing multi-scale feature extraction on the first feature includes:

[0117] Parallelly performing dilated convolution operations with different dilated rates on the first feature to extract features of different scales.

[0118] In this embodiment, the auxiliary branch generates content-adaptive dynamic weights according to the global statistical information of the first feature, weights and fuses the multi-scale features, and outputs the second feature.

[0119] The global context feature vector of the first feature is extracted through global average pooling;

[0120] The global context feature vector is mapped to the same dimension as the number of dilated convolution branches through a fully connected layer;

[0121] The dynamic weight coefficients corresponding to each dilated convolution branch are generated by normalizing through the Softmax function;

[0122] The corresponding multi-scale spatial features are weighted and fused using the dynamic weight coefficients to obtain the second feature.

[0123] In one embodiment, the step of adaptively weighting and fusing by the fusion gating network includes:

[0124] The first feature and the second feature are concatenated in the channel dimension to generate a joint feature;

[0125] The joint feature is input into the convolutional bottleneck network, and an activation function is used on the output of the bottleneck network to generate a pixel-level gating weight map γ∈[0,1];

[0126] According to the gating weight map, an enhanced feature h_final=γ⊙h_main+(1-γ)⊙h_fused is obtained, where h_main is the first feature, h_fused is the second feature, and ⊙ is the Hadamard product.

[0127] This step realizes the dynamic balance of sequence modeling and the contribution weight of spatial perception.

[0128] In one embodiment, the operation of globally aggregating the output feature maps output by the last layer MS-SMamba block in S3 includes inputting the output feature maps into a fully connected layer after processing by a global average pooling layer, and outputting the class probability of the medical image.

[0129] The specific execution process of the auxiliary branch is as follows:

[0130] As shown in Figure 3 , the three parallel branches are:

[0131] Branch-d=1: standard 3x3 depth separable convolution (k=3x3, d=1), output feature h^{(1)}∈R^{B×C×H×W}, effective receptive field 3x3 pixels, capturing local texture details;

[0132] Branch-d=3: 3x3 depthwise separable convolution with dilation rate 3 (k=3x3, d=3), output feature h^{(3)}∈R^{B×C×H×W}, effective receptive field 7x7 pixels, modeling organ-level anatomical structure;

[0133] Branch-d=5: 3x3 depthwise separable convolution with dilation rate 5 (k=3x3, d=5), output feature h^{(5)}∈R^{B×C×H×W}, effective receptive field 11x11 pixels, capturing inter-organ long-range dependencies.

[0134] By controlling the effective receptive field (RF^{(1)}=3, RF^{(3)}=7, RF^{(5)}=11), covering spatial ranges from local to global.

[0135] Through depthwise separable optimization, independent convolution operation is performed on each input channel, reducing the number of parameters (the number of parameters is reduced from C²k² to Ck²). Through 1x1 convolution, multi-channel features are fused to restore the channel dimension. The complexity of standard convolution is O(9HWC) vs. depthwise separable convolution O(3HWC), which is accelerated by 3 times. The computational complexity is reduced from O(C²k²HW) to O(Ck²HW), maintaining linear computing characteristics.

[0136] The cross-scale interaction module (CSI) extracts multi-scale features in parallel and fuses them with the main branch:

[0137]

[0138]

[0139] wherein is a scale weight coefficient (scalar) reflecting the importance of the d-th scale feature, W is a learnable weight matrix generating multi-scale weights, b is a bias vector enhancing the model's expression ability, and Softmax is a normalized exponential function, is the global average pooling output, which compresses the spatial dimension to generate statistical descriptors, is the convolution feature with dilation rate d, d∈{1,3,5} corresponding to local / organ / global scales, is the result of dynamic weight weighted summation, realizing content-aware multi-scale feature integration, enabling the model to automatically adjust different scales according to the feature distribution of the input image.

[0140] This step realizes multi-scale feature extraction and content adaptive fusion.

[0141] As shown in Figure 4 , the dynamic weight weighted summation mechanism realizes content adaptive feature fusion:

[0142] Global Average Pooling g = AvgPool2D(x) compresses spatial dimensions to 1 x C, generating a global feature vector g e R^C. A parameterized fully connected layer f_0(g) = Wg + b (W e R³xC, b e R³) maps g to 3C dimensions (W e R³xC), and Softmax normalization generates probability weights a_d = Softmax(f_0(g))_d, ensuring a probability distribution constraint, a_d e R³, with a_d reflecting the importance of different scale features, a1 favoring local details and a5 favoring global context. The content-adaptive multi-scale fusion feature h_fused = a1 h^{(1)} + a3 h^{(3)} + a5 h^{(5)} is generated by weighting and summing the multi-scale features in the ratio a1:a3:a5. Errors are backpropagated through the chain rule to optimize W, b, and the main branch parameters.

[0143] Softmax function:

[0144]

[0145] where is the i-th element of the output vector, representing the predicted probability of the i-th class, with a value range of [0, 1], z is the input vector (logits) with a dimension of K x 1 (K is the total number of classes), is the i-th element of the input vector z (unnormalized class score), is the result of exponentiating to ensure the output is positive, is the sum of all class exponential values (normalization denominator, ensuring the probability sum is 1), and K is the total number of classes.

[0146] Global Average Pooling output :

[0147]

[0148] where is the input feature map (output from the state space modeling branch), H is the feature map height (Height), and W is the feature map width (Width), is the feature vector at spatial position (i, j), and g is the global average pooling output.

[0149] In one embodiment, the residual connection unit and the feedforward network ensure stable gradient propagation through residual connections, and the feedforward network (MLP Block) enhances nonlinear feature expression:

[0150]

[0151] where is the module input feature (dimension B x (H / 4) x (W / 4) x D), is the core calculation of the MS-SMamba block (including SASF, CSI, dynamic gating, etc. sub-modules), is the output feature after residual connection. The role is to ensure stable gradient propagation and alleviate the gradient vanishing problem in deep network training.

[0152] S3: The output feature map output by the last layer of the MS-SMamba block is aggregated to obtain a global feature vector; and the classification result of the medical image is output according to the global feature vector.

[0153] The specific execution process of S3 is as follows:

[0154] ①The three-dimensional feature map (B x H / 32 x W / 32 x D) is flattened into a two-dimensional matrix (B x (H / 32 x W / 32) x D), which is transposed to (B x D x (H / 32 x W / 32)) after layer normalization, and one-dimensional adaptive global average pooling is performed along the spatial dimension to generate a compact global feature vector (B x D).

[0155] ②Classification based on dynamic gating network fusion of main and auxiliary branch features (B x 2D→B x D):

[0156]

[0157]

[0158]

[0159] where z is the spliced feature, Concat represents splicing, h main is the output of the main branch, carrying global long-range dependency information, h aux is the output of the auxiliary branch, carrying multi-scale spatial perception features, from state space modeling, dim is the dimension identifier, the channel dimension is spliced, the spatial structure is kept intact, z=Concat([h main ,h aux ], dim=1) is equivalent to channel-by-channel splicing, γ is the gating weight (scalar), ranging from 0 to 1, output by the Sigmoid function, σ is the Sigmoid activation function, from multi-scale convolution, RELU is the ReLU activation function, BN is the batch normalization, which accelerates the training convergence, Conv1x1 is the 1x1 convolution, which realizes the channel dimension reduction / upgrade, h final is the dynamic gating network fusion output.

[0160] Sigmoid activation function:

[0161]

[0162] where x is the input value (from the gated convolution output), the unnormalized gating weight, is the output value, compressing the input to the interval [0, 1], representing the probability weight. The main function is to generate pixel-level gating weight γ ∈ [0, 1], realizing the adaptive fusion of the main / auxiliary branch: γ ≈ 1: relying on the main branch (Mamba sequence modeling ability); γ ≈ 0: relying on the auxiliary branch (multi-scale spatial perception ability).

[0163] ReLU activation function:

[0164]

[0165] where x is the input feature value (from the BN layer output), the original feature to be activated, max(0, x) is the output feature value, retaining positive features and suppressing negative values (enhancing non-linear expression ability).

[0166] BN (Batch Normalization):

[0167]

[0168] where x i is the i-th element of the input feature, the original feature to be normalized, m is the batch size, counting the number of samples in the current batch, μ B is the mean of the features in the current batch, centering the feature distribution, is the variance of the features in the current batch, scaling the feature distribution range, is a numerical stability term to prevent the denominator from being zero, is the normalized feature, scaling the feature to zero mean and unit variance, γ, β are scaling factors and translation factors, restoring the feature expression ability (avoiding information loss), y i is the BN layer output, stabilizing the feature distribution, and accelerating the training convergence.

[0169] 1 × 1 convolution (Conv1 × 1):

[0170]

[0171] where is the input feature map, the feature value of input channel c in at position (i, j), the original feature to be processed, is the convolution kernel weight, the connection weight between output channel c out and input channel c in , the linear transformation between feature channels, is the bias term, the bias of output channel c out , enhancing the model expression ability, is the output feature map, the feature value at position (i,j) of output channel c out is the feature after channel dimension transformation, c in is the input channel number, determining the dimension of the weight matrix, c out is the output channel number, determining the dimension of the feature after transformation.

[0172] ③The output feature map is output through a global average pooling layer and a linear classification layer to output the final prediction probability (BxC).

[0173] The following gives an image classification processing logic example of the embodiment of the application (taking a 224x224 resolution as an example):

[0174] ①Input preprocessing: a 224x224x3 medical image is divided into 56x56 4x4 image blocks;

[0175] ②Feature embedding: an initial feature is generated through a 768x(4x4x3) linear projection layer;

[0176] ③Feature extraction: processed through a plurality of MS-SMamba blocks, which can be 12-24 layers;

[0177] ④Feature aggregation: fused through dynamic gating after global average pooling;

[0178] ⑤Classification output: the classification probability is output through a fully connected layer.

[0179] The following describes the specific implementation of MS-SMamba in a medical image classification task:

[0180] (1) Data preprocessing

[0181] Image standardization: the input image is normalized to the range of [0, 1] to eliminate device acquisition differences.

[0182] Data augmentation: random rotation (±15°), elastic deformation (σ=15), and noise injection (Gaussian noise σ=0.1) are used to enhance the generalization ability.

[0183] (2) Model configuration

[0184] Input size: 224x224x3 (supporting multi-scale input, such as 384x384x3).

[0185] Hidden dimension: D=768, the number of channels gradually expands with the depth of the network (256→512→768).

[0186] Classification head: a fully connected layer after global average pooling, outputting multi-classification probability.

[0187] (3) Training strategy

[0188] Optimizer: AdamW optimizer, initial learning rate 1e-4, weight decay 1e-2.

[0189] Loss function: Cross-entropy loss function combined with L2 regularization (λ=1e-4).

[0190] Batch size: 128, gradient accumulation steps: 2 (for memory-constrained scenarios).

[0191] The following visual verification demonstrates the multi-scale modeling capability:

[0192] Tools: Grad-CAM++ was used to generate a heat map of the lesion area.

[0193] Results: MS-SMamba showed improved sensitivity to microcalcifications and improved accuracy in locating tumor margins.

[0194] like Figure 5 As shown, the comparison results of the feature heatmaps are as follows:

[0195] CNN (ResNet50) comparison: Traditional CNNs only activate local regions and cannot connect structures between organs.

[0196] ViT (Swin) comparison: ViT loses details due to patching, and has a weak response to small lesions.

[0197] MS-SMamba: The main branch captures global dependencies, while the auxiliary branches enhance local details, and the heatmap is highly consistent with the clinical gold standard.

[0198] The second aspect of this invention also discloses a medical image classification system based on multi-scale spatial state modeling, according to the first aspect of the invention, comprising:

[0199] The input preprocessing and feature embedding module receives the input medical image and divides it into multiple non-overlapping image patches; it then maps each image patch to a feature space of a preset dimension through a learnable linear projection layer to obtain an initial feature map.

[0200] Multi-layered stacked MS-SMamba block modules are used for layer-by-layer feature extraction from the initial feature map. Each MS-SMamba block includes:

[0201] The main branch is used to perform selective state-space modeling scanning through the state-space model, capture long-range dependencies, and output the first feature.

[0202] The auxiliary branch is used to extract multi-scale features from the first feature and generate content-adaptive dynamic weights based on the global statistical information of the first feature. It then performs weighted fusion of the multi-scale features and outputs the second feature.

[0203] The dynamic gate control fusion network is used for generating a gate weight map according to the first feature and the second feature, adaptively weighting and fusing the first feature and the second feature based on the gate weight map, and obtaining an enhanced feature;

[0204] The residual connection unit and the feedforward network are used for performing residual connection processing on the enhanced feature to obtain an output feature map of a current layer.

[0205] The global feature aggregation and classification module is used for performing global feature aggregation on the output feature map output by the last MS-SMamba block to obtain a global feature vector, and performing classification on the global feature vector to output a classification result of the medical image.

[0206] The second aspect of the embodiment of the present application is used for executing all the steps in the first aspect of the embodiment.

[0207] The present application realizes the following technical breakthroughs in the medical image classification task through the innovative design of the multi-scale space state space modeling and the cross-scale interaction mechanism:

[0208] ① Diagnosis accuracy: the accuracy is improved by 3%-5% on the Kvasir, PAD-UFES-20 and other data sets;

[0209] ②Computing efficiency: the parameter amount is 29M, the inference delay is 5.8ms (batch processing 128), and it is suitable for embedded devices;

[0210] ③Clinical interpretability: the saliency map and the heat map verify the model's attention to the key diagnosis area.

[0211] The present application provides an efficient, reliable and interpretable solution for the medical AI field, and has significant clinical transformation potential.

[0212] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be mutually referred to.

[0213] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A medical image classification method based on multi-scale spatial state modeling, characterized in that, Includes the following steps: S1: Divide the input medical image into multiple non-overlapping image patches; map each image patch to a feature space of a preset dimension through a learnable linear projection layer to obtain an initial feature map; S2: The initial feature map is input into a multi-layered stacked MS-SMamba block for layer-by-layer feature extraction; wherein each MS-SMamba block includes: The main branch is used to perform selective state-space modeling scanning through the state-space model, capture long-range dependencies, and output the first feature. An auxiliary branch is used to extract multi-scale features from the first feature, generate content-adaptive dynamic weights based on the global statistical information of the first feature, perform weighted fusion of the multi-scale features, and output the second feature. A dynamic gated fusion network is used to generate a gated weight map based on the first feature and the second feature, and to adaptively weight and fuse the first feature and the second feature based on the gated weight map to obtain enhanced features; Residual connection units and feedforward networks are used to perform residual connection processing on the enhanced features to obtain the output feature map of the current layer; S3: Perform global feature aggregation on the output feature map of the last layer MS-SMamba block to obtain a global feature vector; classify and output the classification result of the medical image based on the global feature vector.

2. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, In step S1, after mapping each image patch to a feature space of a preset dimension through a learnable linear projection layer to obtain the feature embedding vector, the following steps are also included: By utilizing depthwise separable convolution kernels, local spatial relationships are perceived through convolution based on the feature embedding vectors, and positional codes are dynamically generated and fused, so that the output initial feature map includes spatial positional information.

3. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, The main branch is used to perform the following steps: The input two-dimensional initial feature map is scanned in four directions by a four-way scanner and converted into a one-dimensional sequence. The one-dimensional sequence is input into a selective state-space model for processing to capture long-range dependencies; The output of the selective state-space model is remapped back to two-dimensional space through an expanded convolutional weight reconstructor.

4. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, The auxiliary branch performs multi-scale feature extraction on the first feature, including the following steps: Multiple dilation convolution operations with different dilation rates are performed in parallel on the first feature to extract features at different scales.

5. A medical image classification method based on multi-scale spatial state modeling according to claim 4, characterized in that, The steps of generating content-adaptive dynamic weights based on the global statistical information of the first feature, weighting and fusing multi-scale features, and outputting the second feature include: The global context feature vector of the first feature is extracted by global average pooling; The global context feature vector is mapped to the same dimension as the number of branches in the dilated convolution through a fully connected layer. Normalization is performed using the Softmax function to generate dynamic weight coefficients for each dilated convolution branch. The corresponding multi-scale spatial features are weighted and fused using the dynamic weighting coefficients to obtain the second feature.

6. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, The steps for adaptive weighted fusion in the fusion gating network include: The first feature and the second feature are concatenated along the channel dimension to generate a joint feature; The joint features are input into a convolutional bottleneck network, and an activation function is applied to the output of the bottleneck network to generate a pixel-level gated weight map γ∈[0,1]. Based on the gating weight map, the enhanced feature h_final = γ⊙h_main + (1-γ)⊙h_fused is fused, where h_main is the first feature, h_fused is the second feature, and ⊙ is the Hadamard product.

7. The medical image classification method based on multi-scale spatial state modeling according to claim 1, characterized in that, The operation of global feature aggregation of the output feature map of the last layer MS-SMamba block in S3 includes processing the output feature map by a global average pooling layer and then inputting it into a fully connected layer to output the category probability of the medical image.

8. A medical image classification system based on multi-scale spatial state modeling, according to any one of claims 1-7, characterized in that, include: The input preprocessing and feature embedding module is used to receive the input medical image and divide it into multiple non-overlapping image patches; An initial feature map is obtained by mapping each image patch to a feature space of a preset dimension through a learnable linear projection layer; A multi-layered stacked MS-SMamba block module is used to perform layer-by-layer feature extraction on the initial feature map. Each MS-SMamba block includes: The main branch is used to perform selective state-space modeling scanning through the state-space model, capture long-range dependencies, and output the first feature. An auxiliary branch is used to extract multi-scale features from the first feature, generate content-adaptive dynamic weights based on the global statistical information of the first feature, perform weighted fusion of the multi-scale features, and output the second feature. A dynamic gated fusion network is used to generate a gated weight map based on the first feature and the second feature, and to adaptively weight and fuse the first feature and the second feature based on the gated weight map to obtain enhanced features; Residual connection units and feedforward networks are used to perform residual connection processing on the enhanced features to obtain the output feature map of the current layer; The global feature aggregation and classification module is used to perform global feature aggregation on the output feature map of the last layer MS-SMamba block to obtain a global feature vector; and to perform classification based on the global feature vector to output the classification result of the medical image.

Images