Medical image classification method based on ldm and swin-transformer hierarchical multi-scale feature fusion

Abstract

This invention presents a medical image classification method based on hierarchical multi-scale feature fusion using LDM and Swin-Transformer. First, medical images are acquired and preprocessed. Then, a multi-scale latent feature extraction network based on a pre-trained latent diffusion model framework, a global feature extraction network based on Swin-Transformer, and a medical image classification network based on dual-stream cross-attention and selective channel renormalization are used to obtain fused features and predicted probability distributions. The medical image classification network is then trained, and the learnable parameters in the network are iteratively updated according to the cross-entropy loss function until the loss function converges or a preset training termination condition is met, thus obtaining a trained medical image classification model. This invention not only improves computational efficiency but also enhances the network's ability to integrate multi-scale semantics, thereby achieving more accurate and stable prediction performance in medical image classification tasks.

Description

Technical Field

[0001] This invention belongs to the field of deep learning medical image classification, specifically a medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure, and these statements may constitute prior art. In the process of developing this invention, the inventors discovered at least the following problems in the prior art.

[0003] Medical images encompass multiple modalities, including X-rays, CT scans, MRIs, and ultrasound, providing crucial visual evidence for disease diagnosis and treatment. With the rapid growth in the scale of medical image data, achieving efficient and accurate automatic classification has become a significant research topic in clinical auxiliary diagnosis. Traditional medical image classification largely relies on the experience and judgment of radiologists, which suffers from limitations such as high subjectivity, low efficiency, and susceptibility to fatigue. Therefore, developing artificial intelligence-based automatic classification methods is of urgent significance for improving diagnostic consistency and reducing the burden on physicians.

[0004] In recent years, medical image classification methods, represented by deep learning, have made significant progress. Among them, methods based on convolutional neural networks (CNNs), such as ResNet and DenseNet, have demonstrated excellent performance in modeling local features such as tissue texture and edge structures due to their powerful local feature extraction capabilities. However, CNNs are limited by the receptive field of the fixed-size convolutional kernels, making it difficult to effectively capture long-range dependencies and global contextual information in images, thus limiting their ability to model diffuse lesions and multi-scale anatomical structures commonly found in medical images. On the other hand, self-attention mechanism models, represented by Visual Transformer ViT, can effectively model the semantic relationships between different regions of an image through global attention computation, showing advantages in capturing global context. However, Transformers often neglect local detail features in shallow layers, and the computational complexity of its self-attention mechanism is quadratically related to the image size, resulting in high computational and memory overhead when processing high-resolution medical images, limiting its practical application in resource-constrained environments.

[0005] To integrate the local feature advantages of CNNs with the global modeling capabilities of Transformers, some studies have proposed hybrid architectures, such as TransFuse, GasHis-Transformer, and Medical-Transformer. These methods typically run the CNN and Transformer in two independent branches, interacting with each other through feature concatenation, weighted summation, or attention fusion. For example, patent application number 202410554782.7, entitled "A Medical Image Segmentation Method Based on Feature Interaction," uses a CNN-based and...

[0006] The Transformer's dual-branch encoder performs feature extraction and interaction, preserving local details and global semantic information in medical images to the maximum extent. Prior knowledge is used to guide the learning process stage-by-stage in the boundary-guided decoder. An attention-based feature fusion module is introduced, which uses channel attention to guide effective interaction and fusion of local and global features while suppressing interference from irrelevant background noise. Simultaneously, through the combined use of a multi-scale feature aggregation module and a multi-level feature bridging module, multi-branch strip convolution and multi-level feature interaction are employed to fully mine multi-scale features in medical images. Finally, a weighted loss function is used to perform deep supervised training on the segmentation results of each level, thereby achieving accurate lesion segmentation in medical images.

[0007] However, while such patents have achieved some success in certain medical image segmentation and detection tasks, existing methods still have the following shortcomings:

[0008] (1) Computational redundancy and inefficiency: Most fusion methods need to maintain the complete CNN and Transformer branches in parallel, especially in the shallow feature extraction stage, the computational burden is heavy and the hardware resource requirements are high.

[0009] (2) Simple fusion mechanism: Existing methods mostly adopt explicit fusion strategies such as weighted summation and channel splicing, which fail to fully model the complex nonlinear interaction relationship between global semantics and local details;

[0010] (3) Insufficient cross-level information flow: The fusion process is mostly concentrated on high-level semantic features, lacking progressive, multi-scale semantic guidance from low to high levels, which leads to the gradual loss of detailed information in deep networks, which is not conducive to the refined identification of small lesions and blurred edges in medical images.

[0011] (4) Insufficient utilization of pre-trained models: Current methods rely heavily on CNN or Transformer models pre-trained on natural image datasets such as ImageNet, and fail to effectively utilize the rich visual prior knowledge contained in generative models pre-trained on large-scale multimodal data, such as latent diffusion models.

[0012] To address the aforementioned issues, particularly the challenge of large intra-class variability and small inter-class variability prevalent in medical images, there is an urgent need for a medical image classification method that can efficiently integrate local details with global context, possess cross-level semantic guidance capabilities, and fully utilize advanced pre-trained models to improve classification accuracy, robustness, and generalization ability. Summary of the Invention

[0013] In view of the above problems, the purpose of this invention is to solve some of the problems in the prior art, or at least alleviate these problems.

[0014] A medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion includes the following steps:

[0015] The dimensions to be acquired are The medical images are scaled and preprocessed to obtain a preprocessed medical image sequence. , ;in, Let represent the k-th medical image, where K represents the total number of images in the dataset, h represents the height of the medical image, w represents the width of the medical image, and c represents the number of channels in the medical image. One-hot encoding is used, and each data sample is uniquely identified by a binary vector. The length of the vector is equal to the number of categories, and the position of the "1" corresponds to the true category to which the data sample belongs.

[0016] Based on a pre-trained latent diffusion model framework, a multi-scale latent feature extraction network for medical images is constructed, comprising: a latent encoding module, a latent feature extraction module based on a UNet encoder, and an adaptive feature alignment and channel mapping processing module to map multi-scale latent features to a feature space that matches the subsequent network structure; and a medical image processing network. Inputting the multi-scale latent feature extraction network yields the final latent features. Used for subsequent multi-branch feature fusion; among which... This represents the output of the latent features extracted from the i-th coding layer of the UNet encoder for the k-th medical image, after processing by the adaptive feature and channel adjustment module. ;

[0017] A global feature extraction network based on Swin-Transformer is constructed, comprising: a convolution block module and a four-stage global feature extraction module based on the Swin-Transformer main structure; the global feature extraction module consists of four sequentially connected feature extraction stages; and medical images are processed using this network. The global feature extraction network is input to extract global features. ;in, This represents the global features extracted from the k-th medical image in the i-th stage. ;

[0018] A medical image classification network based on dual-stream cross-attention and selective channel remodeling is constructed, comprising: a four-stage feature fusion module and a classification module, with dual-stream cross-attention and selective channel remodeling as the main structure. The feature fusion module consists of four fusion layers. The first stage is used to establish initial semantic alignment between global features and latent features. The second to fourth stages, based on the features fused in the first stage, introduce a cross-layer semantic guidance mechanism to achieve multi-scale and multi-level adaptive fusion. The global features are then... Potential characteristics Input the medical image classification network to obtain fused features and predicted probability distribution ;in This represents the fusion features extracted from the k-th medical image at the i-th layer. ; This represents the probability that the model predicts the k-th sample as belonging to the n-th class.

[0019] A loss function is constructed for training the medical image classification network; the loss function adopts the cross-entropy loss function to measure the difference between the model's prediction results and the true class labels, and its expression is:

[0020]

[0021] Where K represents the total number of training samples, and N represents the total number of disease categories. This represents the true label value of the k-th image sample in the n-th category;

[0022] A gradient descent-based optimization algorithm is used to train a medical image classification network. The learnable parameters in the network are iteratively updated according to the cross-entropy loss function until the loss function converges or a preset training termination condition is met, thereby obtaining a trained medical image classification model. The preferred optimization algorithm is the Adam optimization algorithm.

[0023] The latent diffusion model uses a pre-trained autoencoder (VAE) and a U-shaped neural network (UNet) encoder as the backbone network for feature extraction, and their parameters remain fixed during the feature extraction stage; medical images are then processed. Inputting the multi-scale latent feature extraction network yields the final latent features. Including: Compressed into latent space It is fed into the UNet encoder, which outputs four stages of latent features. And then Process to output the final latent features Specifically, the steps are as follows:

[0024] For the input two-dimensional medical image Perform latent encoding to map it from pixel space to a latent representation in a low-dimensional latent space. ;

[0025] Will The input is fed into a pre-trained UNet encoder; the encoding path of the UNet encoder consists of four downsampling encoding layers, each containing several residual blocks, a downsampling module, and an optional attention module; latent features with different spatial resolutions and semantic levels are output from each encoding layer to obtain multi-scale latent features. ,in This represents the latent features extracted from the k-th medical image in the i-th coding layer of the UNet encoder;

[0026] Multi-scale latent features output by the UNet encoder Adaptive feature alignment and channel mapping are performed to obtain the latent feature representation of the target dimension. ,in This represents the output of the latent features extracted from the i-th coding layer of the UNet encoder for the k-th medical image, after processing by the adaptive feature and channel adjustment module, which is used for subsequent multi-branch feature fusion.

[0027] Furthermore, the input two-dimensional medical image Perform latent encoding to map it from pixel space to a latent representation in a low-dimensional latent space. This includes the following steps:

[0028] Will Normalization is performed, and the input image is resized to the input size requirement of the pre-trained autoencoder VAE using bilinear interpolation to obtain the image. ;

[0029] Will The input is fed into the initial convolutional layer of the VAE encoder, which maps the number of input channels to the feature space to obtain the initial feature representation. ;

[0030] Represent the initial features The features are sequentially input into multiple levels of encoding modules for feature compression. Each encoding module consists of several residual blocks and a downsampling module. By progressively reducing the spatial resolution of the feature map and increasing the channel dimension, intermediate feature representations at different scales are obtained. The output feature representation of each encoding level includes: the output features of the first encoding layer. The second coding layer outputs features The third coding layer outputs features The third encoding layer is the last encoding layer of the VAE encoder. A self-attention module is introduced at this layer to enhance the ability of latent features to model the global structural information of medical images, thereby obtaining feature representations. ;

[0031] At the bottleneck layer of the encoder, for features Further integration and semantic enhancement are performed, and the bottleneck layer outputs feature representations. ;

[0032] Will Channel compression is performed through convolutional mapping layers to output the latent representation. This is used for multi-scale feature extraction in the subsequent UNet encoder.

[0033] Furthermore, Inputting the data into a pre-trained UNet encoder yields multi-scale latent features. This includes the following steps:

[0034] potential representation The input is fed into the initial encoding layer of the UNet encoder, which maps the number of latent channels to the basic channel dimension of UNet to obtain the initial latent feature representation. ;

[0035] Will The latent features are sequentially fed into multiple downsampling coding layers of the UNet encoder for hierarchical feature modeling. In each coding layer, local feature modeling and semantic enhancement are first performed on the input latent features through residual blocks. Then, the spatial resolution of the feature map is reduced and the number of feature channels is increased through a downsampling module. Attention modules are injected into the fourth coding layer, thereby progressively improving the semantic abstraction level of the latent features. Specifically, latent features are extracted in the first coding layer of the UNet encoder. The second encoding layer extracts latent features. The third encoding layer extracts latent features. The fourth coding layer extracts latent features. .

[0036] Multi-scale latent features output by the UNet encoder Adaptive feature alignment and channel mapping are performed to obtain the latent feature representation of the target dimension. This includes the following steps:

[0037] right Perform upsampling, Adjust the feature resolution to the corresponding dimension of feature fusion;

[0038] Based on the convolutional channel mapping unit, the channels of the result from the previous step are adjusted to the corresponding dimensions to obtain new feature representations;

[0039] The results from the previous steps are then normalized and enhanced with nonlinear activation units to obtain latent features. This is used for subsequent multi-branch feature fusion.

[0040] Medical images The global feature extraction network is input to extract global features. This includes the following steps:

[0041] The convolution block module uses convolutional layers and linear layers sequentially... The k-th feature map is obtained by performing block embedding, projection, and flattening. ;

[0042] The global feature extraction module The four-stage encoding process is performed step by step to obtain the results. , , , This constitutes a multi-scale global feature representation; specifically, each of the four feature extraction stages contains two consecutive SwingTransformer blocks, with the first stage directly... Process the data to output the first layer of global features. A downsampling module is set before the second, third, and fourth stages to halve the spatial size of the input feature map and double the number of channels. For the i-th stage (i=2,3,4): the output feature map of the (i-1)-th stage is... The data is sequentially processed by the downsampling module and two consecutive SwinTransformer blocks, and the global features extracted in the i-th stage are output. The feature representations are obtained sequentially. , , ;

[0043] Each SwinTransformer block employs a structure that alternates between multi-head window attention and shifted window attention, with window_size set to 7.

[0044] The downsampling module processes the input feature map (i=2,3,4) Process according to the following steps:

[0045] Spatial resampling: Sampling is performed in the height and width dimensions of the feature map with a stride of 2, resulting in four dimensions. × × Sub-feature maps;

[0046] Channel stitching: The four sub-feature maps mentioned above are stitched together along the channel dimension to obtain a size of... × × Intermediate features;

[0047] Layer normalization: Perform layer normalization on the concatenated intermediate features;

[0048] Linear projection: Transforming the number of channels from 4 using a linear transformation. Reduced to 2 This allows for a halving of the feature map space size, doubling the number of channels, and outputting the result.

[0049] in, Indicates the height of the sub-feature map, Indicates the width of the sub-feature map. This represents the number of channels in the sub-feature map.

[0050] The global features Potential characteristics Input the medical image classification network to obtain fused features and predicted probability distribution This includes the following steps:

[0051] In the first stage, the feature fusion module only uses global features. With latent features As input, the first-stage fused features are output through a window-based dual-stream cross-attention mechanism. ;

[0052] In the second, third, and fourth stages, the feature fusion module of the i-th layer uses the global features of the i-th layer. Potential characteristics and the fusion features of the (i-1)th layer As input, Through cross-layer semantic guidance, dual-stream cross-attention, phased cascaded fusion, and selective channel reorganization, the following results are obtained sequentially. , , ;

[0053] The classification module uses fourth-layer fusion features. As input, output image Predicted probability of belonging to each disease category , Where n represents the nth type of disease; the classification module includes a global average pooling module, a layer normalization module and a fully connected classification module connected in sequence, used to classify the fourth layer fused features.

[0054] Furthermore, the i-th layer feature fusion module uses the global features of the i-th layer. Potential characteristics and the fusion features of the (i-1)th layer As input, the process includes the following steps: cross-layer semantic guidance, dual-stream cross-attention, phased cascaded fusion, and selective channel reshaping.

[0055] Fusion features of the (i-1)th layer Channel alignment and downsampling are performed to obtain a feature representation consistent with the feature space of the i-th layer. ,right Global average pooling and multilayer perceptron mapping are performed to generate cross-layer semantic guidance vectors. ;

[0056] and exist Under the channel-level modulation, respectively, the following were obtained: , and to and Window-based two-stream cross-attention computation is performed to obtain the fused cross-modal features. ;

[0057] Features Integration characteristics with the previous stage The features are concatenated along the channel dimension and further concatenated with the original global features and latent features of the current layer through multi-source concatenation. This concatenation is used to introduce multi-source semantic information to form a multi-source concatenated feature representation. ;in, This represents the number of channels in the multi-source splicing feature representation of the i-th layer.

[0058] Features Perform Selective Channel Reforming (SCR-MLP). Perform a global average pooling operation in the spatial dimension and input it into a multilayer perceptron for nonlinear mapping to obtain... Then change the channel weight vector Acting on the original splicing features Obtain the weighted feature map ,right Nonlinear mapping can output ;

[0059] The reshaped feature representation After channel alignment using 1×1 convolution, and... Residual connections are performed to maintain the continuity of features across stages and ensure stable training, resulting in the final fused features of the i-th layer. .

[0060] Furthermore, the calculation of the dual-stream cross-attention of the window includes the following steps:

[0061] Global features of the i-th layer and potential characteristics Layer normalization is performed separately to eliminate differences in the distribution of different features and stabilize feature representation;

[0062] The normalized latent and global feature representations are divided into windows with a window size of 7. Within each local window, a bidirectional cross-attention mechanism is constructed, which simultaneously "guides global features" and "reverses constraints latent features" to achieve symmetrical information interaction and structural alignment between different semantic spaces. Specifically, within each window, the attention response of latent features to global semantic information is calculated using latent features as the query vector and global features as the key and value vectors. At the same time, the attention response of global features to latent fine-grained structural information is calculated using global features as the query vector and latent features as the key and value vectors.

[0063] The bidirectional cross-attention generates queries, keys, and values ​​through a learnable linear mapping, and independently calculates attention weights and performs weighted feature aggregation within each window to obtain enhanced latent feature representations and global feature representations, as shown in the following formulas:

[0064]

[0065]

[0066] in Representing global features respectively Query, key, value They represent latent features respectively. The query, key, and value are T, where T represents the transpose operation of a matrix, and d represents the feature dimension of a vector.

[0067] The attention results obtained in each window are restored to the original spatial layout through inverse window rearrangement, and residual connections are made with the corresponding input features to obtain the latent feature representation enhanced by cross-modal semantic interaction. , Finally, the two are concatenated along the channel dimension to form a block modal interaction feature representation. ;in , , These represent the number of channels, height, and width of the fused feature in the i-th layer, respectively.

[0068] Furthermore, the classification module uses a fourth layer of fused features. As input, output image Predicted probability of belonging to each disease category This includes the following steps:

[0069] The fourth layer of fusion features The input is fed into the global average pooling module, which performs an average operation on the spatial dimensions corresponding to each channel to obtain a channel-level global semantic feature representation.

[0070] The global semantic feature representation is input into the layer normalization module to normalize the channel features of each sample;

[0071] The features, after being processed by layer normalization, are input into a fully connected classification layer. The channel features are linearly mapped to the disease category space, and the image is output through a softmax activation function. Predicted probability distribution of each disease category .

[0072] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion.

[0073] The present invention has the following beneficial effects:

[0074] 1. This invention uses a pre-trained latent diffusion model to extract latent spatial features, replacing the traditional CNN's calculation in pixel space, which significantly reduces the computational burden and memory consumption of shallow feature extraction; combined with the efficient window-based attention mechanism of Swin-Transformer, it achieves effective modeling of long-range dependencies in medical images under limited computing resources.

[0075] 2. This invention designs a window-based dual-stream cross-attention fusion mechanism, which realizes bidirectional and symmetrical interaction between global semantics and local details within the local receptive field. This mechanism can adaptively integrate features at different scales, enhance the modeling of subtle lesion structures and blurred tissue boundaries in medical images, and effectively alleviate the problem of detail loss in deep networks;

[0076] 3. This invention innovatively introduces a cross-layer semantic guidance mechanism, which dynamically modulates the features of the current stage using the output of the previous fusion stage. This top-down semantic guidance establishes a continuous information flow from high-level abstract semantics to low-level specific features, enhances the semantic consistency between features at different levels, and improves the model's ability to understand the hierarchical structure of complex medical images.

[0077] 4. This invention employs a selective channel reweighting module to adaptively reweight the features after multi-source fusion at the channel level. This module automatically strengthens feature channels with strong discriminative power for classification tasks and suppresses redundant or noisy channels, thereby improving the purity and discriminative power of the final fused features, making classification decisions more robust and accurate.

[0078] 5. This invention directly utilizes an LDM model pre-trained on a large-scale general dataset as the feature extraction backbone, transferring its powerful visual representation capabilities to the field of medical images. This method reduces the dependence on large-scale labeled medical data, enabling the model to maintain good feature extraction and generalization capabilities even in clinical scenarios with limited data.

[0079] Through the synergistic effect of the above-mentioned innovative methods, this invention solves the problem of large intra-class differences and small inter-class differences that are common in medical images. It is a medical image classification method that can efficiently integrate local details and global context, has cross-level semantic guidance capabilities, and can make full use of advanced pre-trained models, thereby improving the accuracy, robustness and generalization ability of classification. Attached Figure Description

[0080] Figure 1 This is an organizational chart of the present invention;

[0081] Figure 2 This is a flowchart of the fusion module of the present invention. Detailed Implementation

[0082] The present invention will be further described below with reference to the accompanying drawings. The embodiments of the present invention are only used to illustrate the present invention and not to limit the present invention. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the technical concept of the present invention should be included within the scope of the present invention.

[0083] This invention addresses the common problems of large intra-class differences and small inter-class differences in existing medical images. It proposes a multi-branch, multi-scale feature fusion network that combines a latent diffusion model with the Swin-Transformer to solve the problems of high computational complexity and insufficient semantic information interaction in existing technologies. This aims to achieve accurate medical image recognition and classification and contribute to downstream tasks in medical image processing.

[0084] The technical solution adopted in this invention is a medical image classification method based on a multi-branch, multi-scale feature fusion network of LDM and Swin-Transformer, such as... Figure 1 As shown, this medical image classification method includes the following steps:

[0085] Step 1: Obtain the dimension as The medical images are scaled and preprocessed to obtain a preprocessed medical image sequence. , .in, Let represent the k-th medical image, where K represents the total number of images in the dataset, h=224 represents the height of the medical image, w=224 represents the width of the medical image, and c=3 represents the number of channels of the medical image. One-hot encoding is used, and each data sample is uniquely identified by a binary vector. The length of the vector is equal to the number of categories, and the position of the "1" corresponds to the true category to which the data sample belongs.

[0086] Step 2: Based on the pre-trained latent diffusion model framework, construct a multi-scale latent feature extraction network for medical images, including: a latent encoding module, a latent feature extraction module based on a UNet encoder, and an adaptive feature alignment and channel mapping processing module to map multi-scale latent features to a feature space that matches the subsequent network structure. Compressed into latent space It is fed into the UNet encoder, which outputs four stages of latent features. And then Process and output the final latent representation Used for subsequent multi-branch feature fusion, where This represents the output of the latent features extracted from the i-th encoding layer of the UNet encoder in the k-th medical image, after processing by the adaptive feature and channel adjustment module. The latent diffusion model uses a pre-trained autoencoder (VAE) and a U-shaped neural network (UNet encoder) from StableDiffusionv1.4 as the backbone network for feature extraction, and its parameters remain fixed during the feature extraction stage.

[0087] Step 2.1: Process the input two-dimensional medical image. Perform latent encoding to map it from pixel space to a latent representation in a low-dimensional latent space. .

[0088] Step 2.1.1, The input image is normalized and then resized to 256×256×3 using bilinear interpolation to meet the input size requirements of the pre-trained autoencoder (VAE), resulting in the pre-processed image. ;

[0089] Step 2.1.2, will The input is fed into the initial convolutional layer of the VAE encoder, where a 3×3 convolution maps the number of input channels to a 128-dimensional feature space to obtain the initial feature representation. Its spatial resolution is 256×256;

[0090] Step 2.1.3: Represent the initial features The features are sequentially input into multiple levels of encoding modules for feature compression. Each encoding module consists of several residual blocks and a downsampling module. By progressively reducing the spatial resolution of the feature map and increasing the channel dimension, intermediate feature representations at different scales are obtained. Specifically, the output feature representations of each encoding level include:

[0091] The first coding layer outputs features. The second coding layer outputs features The third coding layer outputs features ;

[0092] The 32×32 resolution coding layer is the last coding layer of the VAE encoder. A self-attention module is introduced at this layer to enhance the ability of latent features to model the global structural information of medical images and obtain feature representations. .

[0093] Step 2.1.4: In the bottleneck layer of the encoder, for the features... Further integration and semantic enhancement are performed, and the bottleneck layer outputs feature representations. ;

[0094] Step 2.1.5, Channel compression is performed through convolutional mapping layers to output the latent representation. This is used for multi-scale feature extraction in the subsequent UNet encoder;

[0095] Where 4 represents the number of basic channels in the latent space of the latent diffusion model, used to maintain structural compatibility with the pre-trained UNet encoder. Used to expand and adjust the potential channel capacity.

[0096] Step 2.2, The input is fed into a pre-trained UNet encoder, whose encoding path consists of four downsampling encoding layers. Each encoding layer contains several residual blocks, a downsampling module, and an optional attention module. Latent features with different spatial resolutions and semantic levels are output from each encoding layer to obtain... ,in This represents the latent features extracted from the k-th medical image in the i-th coding layer of the UNet encoder.

[0097] Step 2.2.1: Transform the latent representation The input is fed into the initial encoding layer of the UNet encoder, where a 3×3 convolution maps the number of latent channels to the basic channel dimension of UNet, resulting in the initial latent feature representation. , This represents the base number of channels for the UNet encoder.

[0098] Step 2.2.2: The latent features after initial mapping are sequentially input into multiple downsampling coding layers of the UNet encoder for hierarchical feature modeling. In each coding layer, local feature modeling and semantic enhancement of the input latent features are first performed through residual blocks. Then, the spatial resolution of the feature map is reduced and the number of feature channels is increased through the downsampling module. At the same time, the attention module is injected at the 4×4 resolution level, thereby progressively improving the semantic abstraction level of the latent features.

[0099] Specifically, latent features are extracted in the first encoding layer of the UNet encoder. The second encoding layer extracts latent features. The third encoding layer extracts latent features. The fourth coding layer extracts latent features. .

[0100] Step 2.3: Analyze the multi-scale latent features output by the UNet encoder. Adaptive feature alignment and channel mapping are performed to obtain the latent feature representation of the target dimension. ,in This represents the output of the latent features extracted from the i-th coding layer of the UNet encoder for the k-th medical image, after processing by the adaptive feature and channel adjustment module, which is used for subsequent multi-branch feature fusion.

[0101] Step 2.3.1, for Perform upsampling, The feature resolutions were adjusted to 56×56, 28×28, 14×14, and 7×7, respectively.

[0102] Step 2.3.2: Based on the 1×1 convolutional channel mapping unit, adjust the channels of the result in Step 2.3.1 to 96, 192, 384, and 768 respectively to obtain new feature representations;

[0103] Step 2.3.3: Obtain latent features by normalizing and enhancing feature stability using nonlinear activation units, based on the results of Step 2.3.2. This is used for subsequent multi-branch feature fusion.

[0104] Step 3: Construct a global feature extraction network based on Swin-Transformer, including: a convolution block module and a four-stage global feature extraction module based on the Swin-Transformer main structure. This global feature extraction module consists of four sequentially connected feature extraction stages, each extracting global features. ,in This represents the global features extracted from the k-th medical image in the i-th stage. .

[0105] Step 3.1: The convolution block module uses convolutional layers and linear layers sequentially to... The k-th feature map is obtained by performing block embedding, projection, and flattening. Setting the kernel_size and stride of the convolution block module to 4 reduces the image size by a factor of 4, and using 96 convolution kernels expands the number of channels to 96 to obtain feature maps. .

[0106] Step 3.2, the global feature extraction module... The four-stage encoding process is performed step by step to obtain the results. , , , This constitutes a multi-scale global feature representation.

[0107] Step 3.2.1: Each of the four feature extraction stages contains two consecutive SwingTransformer blocks. The first stage directly... The inputs are sequentially fed into two SwinTransformer blocks, keeping the feature size and number of channels constant, and the first layer of global features is output. ;

[0108] Step 3.2.2, and before stages two, three, and four, a downsampling module is set up to halve the spatial size of the input feature map and double the number of channels. For stage i (i=2,3,4): the output feature map of stage i-1 is... The data is sequentially processed by the downsampling module and two consecutive SwinTransformer blocks, and the global features extracted in the i-th stage are output. The feature representations are obtained sequentially. , , .

[0109] Each SwinTransformer block employs a structure that alternates between multi-head window attention and shifted window attention, with window_size set to 7.

[0110] Among them, the Patch-merging downsampling module processes the input feature map. (i=2,3,4) Process according to the following steps:

[0111] (1) Spatial resampling: Sampling is performed in the height and width dimensions of the feature map with a stride of 2, resulting in four dimensions. × × Sub-feature maps;

[0112] (2) Channel stitching: The above four sub-feature maps are stitched together along the channel dimension to obtain a size of × × Intermediate features;

[0113] (3) Layer normalization: Perform layer normalization on the spliced ​​intermediate features;

[0114] (4) Linear projection: The number of channels is reduced from 4 through linear transformation. Reduced to 2 This allows for a halving of the feature map space size, doubling the number of channels, and outputting the result.

[0115] in, Indicates the height of the sub-feature map, Indicates the width of the sub-feature map. This represents the number of channels in the sub-feature map.

[0116] Step 4: Construct a medical image classification network based on dual-stream cross-attention and selective channel remodeling. In this example, the image feature fusion module, such as... Figure 2 As shown, it includes: a four-stage feature fusion module with dual-stream cross-attention and selective channel reconfiguration as its main structure, and a classification module. The feature fusion module consists of four fusion layers. The first stage establishes initial semantic alignment between global features and latent features. The second to fourth stages, based on the features fused in the first stage, introduce a cross-layer semantic guidance mechanism to achieve multi-scale, multi-level adaptive fusion, resulting in... ,in This represents the fusion features extracted from the k-th medical image at the i-th layer. .

[0117] Step 4.1, First stage: The feature fusion module uses only global features. With latent features As input, the feature information is interacted with and semantically aligned through a window-based dual-stream cross-attention mechanism, and the first-stage fused features are output. .

[0118] Step 4.2, the second, third, and fourth stages, the i-th layer feature fusion module uses the global features of the i-th layer. Potential characteristics and the fusion features of the (i-1)th layer As input, Through cross-layer semantic guidance, dual-stream cross-attention, phased cascaded fusion, and selective channel reorganization, the following results are obtained sequentially. , , .

[0119] Step 4.2.1: Combine the fused features of layer i-1 using 1×1 convolution and average pooling. Channel alignment and downsampling are performed to obtain a feature representation consistent with the feature space of the i-th layer. ,right Global average pooling is performed in the spatial dimension, and cross-layer semantic guidance vectors are generated through a multilayer perceptron and a sigmoid activation function. .

[0120] in , , These represent the number of channels, height, and width of the fused feature in the i-th layer, respectively.

[0121] Step 4.2.2 and Cross-layer semantic guidance vectors Under the channel-level modulation, respectively, the following were obtained , ;right and Window-based two-stream cross-attention computation is performed to obtain the fused cross-modal features. ;

[0122] The calculation of the dual-stream cross-attention W-MCA within the window includes the following steps:

[0123] (1) Global features of the i-th layer and potential characteristics Layer normalization is performed separately to eliminate differences in the distribution of different features and stabilize feature representation;

[0124] (2) The normalized latent and global feature representations are divided into windows, and the window_size is set to 7. Within the local window, a bidirectional cross-attention mechanism of "latent features guiding global features" and "global features inversely constraining latent features" is constructed simultaneously to achieve symmetrical information interaction and structural alignment between different semantic spaces.

[0125] (3) Specifically, within each window, using latent features as the query vector and global features as the key and value vectors, the attention response of latent features to global semantic information is calculated; simultaneously, using global features as the query vector and latent features as the key and value vectors, the attention response of global features to latent fine-grained structural information is calculated, thus forming a bidirectional semantic interaction mechanism. The bidirectional cross-attention generates queries, keys, and values ​​through learnable linear mappings, and independently completes attention weight calculation and feature weighted aggregation within each window to obtain enhanced latent feature representations and global feature representations, as shown in the following formulas:

[0126]

[0127]

[0128] in Representing global features respectively Query, key, value They represent latent features respectively. The query, key, and value are T, where T represents the transpose operation of a matrix, and d represents the feature dimension of a vector.

[0129] (4) The attention results obtained in each window are restored to the original spatial layout by inverse window rearrangement, and residual connections are made with the corresponding input features to obtain the latent feature representation enhanced by cross-modal semantic interaction. , Finally, the two are concatenated along the channel dimension to form a block modal interaction feature representation. .

[0130] Step 4.2.3, Features Integration characteristics with the previous stage Concatenate along the channel dimension to introduce cross-layer contextual semantic information, and combine the concatenated result with the original global features of the current layer. and potential characteristics Further concatenation along the channel dimension is performed to supplement the multi-source detail information of the current layer, resulting in a multi-source spliced ​​feature representation. ;

[0131] in This represents the number of channels in the multi-source splicing feature representation of the i-th layer.

[0132] Step 4.2.4: Add features Selective Channel Restructuring (SCR-MLP) is performed to restructure cross-layer, multi-source cascaded features at the channel level, addressing the semantic redundancy and channel conflicts introduced by direct concatenation of multi-source features. Perform a global average pooling operation in the spatial dimension and input it into a multilayer perceptron for nonlinear mapping to obtain... Then change the channel weight vector Acting on the original splicing features Obtain the weighted feature map ,right Nonlinear mapping can output ;

[0133] Step 4.2.5: Represent the features after reshaping the features in Step 4.2.4. After channel alignment using 1×1 convolution, and... Residual connections are performed to maintain the continuity of features across stages and ensure stable training, resulting in the final fused features of the i-th layer. .

[0134] Step 4.3: The classification module includes a global average pooling module, a layer normalization module, and a fully connected classification module connected in sequence. This module is used to classify the fourth-layer fused features. As input, output image Predicted probability of belonging to each disease category , , where n represents the disease of the nth category.

[0135] Step 4.3.1: Merge the features of the fourth layer. The input is fed into the global average pooling module, which performs an average operation on the spatial dimensions corresponding to each channel to obtain a channel-level global semantic feature representation, which is used to eliminate the influence of spatial location information on the classification results and enhance global discriminability;

[0136] Step 4.3.2: Input the global semantic feature representation into the layer normalization module to normalize the channel features of each sample in order to stabilize the feature distribution and improve the stability and convergence speed of model training.

[0137] Step 4.3.3: Input the features after layer normalization into the fully connected classification layer, linearly map the channel features to the disease category space, and output the image through the softmax activation function. Predicted probability distribution of each disease category .

[0138] Step 5: Construct a loss function for training the medical image classification network. The loss function is the cross-entropy loss function, which measures the difference between the model's prediction results and the true class labels. Its expression is:

[0139]

[0140] Where K represents the total number of training samples, and N represents the total number of disease categories. This represents the true label value of the k-th image sample in the n-th category. This represents the probability that the model predicts the k-th sample as belonging to the n-th class.

[0141] The cross-entropy loss function is used to supervise and constrain the output of the classification module, guiding the feature fusion module and the classification module in the network to optimize collaboratively.

[0142] Step 6: Train the medical image classification network using a gradient descent-based optimization algorithm. Iteratively update each learnable parameter in the network according to the cross-entropy loss function described in Step 5 until the loss function converges or a preset training termination condition is met, thereby obtaining a trained medical image classification model for the task of classifying medical images. In this embodiment, the optimization algorithm is preferably the Adam optimization algorithm.

[0143] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion.

[0144] To address the challenges of large intra-class variability, small inter-class variability, and scarce labeled data in medical images, this invention proposes a medical image classification method based on hierarchical multi-scale feature fusion using LDM and Swin-Transformer. This method aims to fully exploit the discriminative information of medical images at different semantic levels, thereby improving classification accuracy and robustness. The designed feature fusion network comprises three complementary branch structures: a hierarchical window attention mechanism based on Swin-Transformer, which captures global contextual information of the image while maintaining computational efficiency; a latent diffusion model, which maps the original image to a compact latent space where it learns the implicit semantic representation of the image; and a multi-scale feature fusion module, which hierarchically fuses large-scale semantic information from the global branch with the latent deep features from the diffusion branch.

[0145] This invention extracts multi-scale latent features using a pre-trained Latent Diffusion Model (LDM) to replace traditional CNN branches, significantly reducing computational overhead while preserving rich semantic information. Furthermore, it combines the powerful global modeling capabilities of the Swin-Transformer to construct a dual-branch feature extraction network. By designing a window-based dual-stream cross-attention fusion mechanism, cross-layer semantic guidance, and a selective channel reshaping strategy, it achieves adaptive and refined fusion of global and local features from shallow to deep layers, enabling accurate identification and classification of complex anatomical structures and lesion regions in various modalities of medical images such as X-rays, CT scans, and MRIs.

[0146] This invention characterizes the key structural information and implicit semantic features of medical images, compressing the images into a latent space for feature modeling. This not only improves computational efficiency but also enhances the network's ability to integrate multi-scale semantics, thereby achieving more accurate and stable prediction performance in medical image classification tasks. This invention particularly emphasizes the preservation and utilization of subtle structural features in images, effectively improving the model's classification accuracy, robustness, and generalization ability under limited data conditions, providing an efficient and reliable technical tool for clinical auxiliary diagnosis.

[0147] The feature fusion network in this invention integrates global feature extraction branches and latent feature extraction branches, and models global-latent multi-scale features in the dataset. The global path and latent diffusion path do not interact on their respective branches to maintain the global feature representation at each level mapped from the original image space. This results in a stronger ability to extract global and latent features from different levels, making it more suitable for medical datasets. Furthermore, this invention proposes a novel general-purpose backbone network for medical image classification, which is expected to contribute to various downstream tasks in the field of medical imaging.

Claims

1. A medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion, characterized in that, Includes the following steps: The dimensions to be acquired are The medical images are scaled and preprocessed to obtain a preprocessed medical image sequence. , ;in, Let represent the k-th medical image, where K represents the total number of images in the dataset, h represents the height of the medical image, w represents the width of the medical image, and c represents the number of channels in the medical image; one-hot encoding is used, and each data sample is uniquely identified by a binary vector. Based on a pre-trained latent diffusion model framework, a multi-scale latent feature extraction network for medical images is constructed, comprising: a latent encoding module, a latent feature extraction module based on a UNet encoder, and an adaptive feature alignment and channel mapping processing module to map multi-scale latent features to a feature space that matches the subsequent network structure; and a medical image processing network. Inputting the multi-scale latent feature extraction network yields the final latent features. ;in, This represents the output of the latent features extracted from the i-th coding layer of the UNet encoder for the k-th medical image, after processing by the adaptive feature and channel adjustment module. ; A global feature extraction network based on Swin-Transformer is constructed, comprising: a convolution block module and a four-stage global feature extraction module based on the Swin-Transformer main structure; the global feature extraction module consists of four sequentially connected feature extraction stages; and medical images are processed using this network. The global feature extraction network is input to extract global features. ;in, This represents the global features extracted from the k-th medical image in the i-th stage. ; A medical image classification network based on dual-stream cross-attention and selective channel remodeling is constructed, comprising: a four-stage feature fusion module and a classification module with dual-stream cross-attention and selective channel remodeling as the main structure; the feature fusion module consists of four fusion layers, wherein the first stage is used to establish initial semantic alignment between global features and latent features, and the second to fourth stages, based on the feature fusion in the first stage, introduce a cross-layer semantic guidance mechanism; the global features are then... Potential characteristics Input the medical image classification network to obtain fused features and predicted probability distribution ;in This represents the fusion features extracted from the k-th medical image at the i-th layer. ; This represents the probability that the model predicts the k-th sample as belonging to the n-th class. A loss function is constructed for training the medical image classification network; the loss function adopts the cross-entropy loss function, and its expression is: Where K represents the total number of training samples, and N represents the total number of disease categories. This represents the true label value of the k-th image sample in the n-th category; The medical image classification network is trained by iteratively updating each learnable parameter in the network according to the cross-entropy loss function until the loss function converges or the preset training termination condition is met, thereby obtaining the trained medical image classification model.

2. The medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion according to claim 1, characterized in that, The latent diffusion model uses a pre-trained autoencoder (VAE) and a U-shaped neural network (UNet) encoder as the backbone network for feature extraction, and their parameters remain fixed during the feature extraction stage; medical images are then processed. Inputting the multi-scale latent feature extraction network yields the final latent features. Including: Compressed into latent space It is fed into the UNet encoder, which outputs four stages of latent features. And then Process to output the final latent features Specifically, the steps are as follows: For the input two-dimensional medical image Perform latent encoding to map it from pixel space to a latent representation in a low-dimensional latent space. ; Will The input is fed into a pre-trained UNet encoder; the encoding path of the UNet encoder consists of four downsampling encoding layers, each containing several residual blocks, a downsampling module, and an optional attention module; latent features with different spatial resolutions and semantic levels are output from each encoding layer to obtain multi-scale latent features. ,in This represents the latent features extracted from the k-th medical image in the i-th coding layer of the UNet encoder; Multi-scale latent features output by the UNet encoder Adaptive feature alignment and channel mapping are performed to obtain the latent feature representation of the target dimension. ,in This represents the output result of the latent features extracted from the i-th coding layer of the UNet encoder for the k-th medical image, after processing by the adaptive feature and channel adjustment module.

3. The medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion according to claim 2, characterized in that, For the input two-dimensional medical image Perform latent encoding to map it from pixel space to a latent representation in a low-dimensional latent space. This includes the following steps: Will Normalization is performed, and the input image is resized to the input size requirement of the pre-trained autoencoder VAE using bilinear interpolation to obtain the image. ; Will The input is fed into the initial convolutional layer of the VAE encoder, which maps the number of input channels to the feature space to obtain the initial feature representation. ; Represent the initial features The features are sequentially input into multiple levels of encoding modules for feature compression. Each encoding module consists of several residual blocks and a downsampling module. By progressively reducing the spatial resolution of the feature map and increasing the channel dimension, intermediate feature representations at different scales are obtained. The output feature representation of each encoding level includes: the output features of the first encoding layer. The second coding layer outputs features The third coding layer outputs features The third encoding layer is the last encoding layer of the VAE encoder. A self-attention module is introduced at this layer to enhance the ability of latent features to model the global structural information of medical images, thereby obtaining feature representations. ; At the bottleneck layer of the encoder, for features Further integration and semantic enhancement are performed, and the bottleneck layer outputs feature representations. ; Will Channel compression is performed through convolutional mapping layers to output the latent representation. .

4. The medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion according to claim 2, characterized in that, Will Inputting the data into a pre-trained UNet encoder yields multi-scale latent features. This includes the following steps: potential representation The input is fed into the initial encoding layer of the UNet encoder, which maps the number of latent channels to the basic channel dimension of UNet to obtain the initial latent feature representation. ; Will The latent features are sequentially fed into multiple downsampling coding layers of the UNet encoder for hierarchical feature modeling. In each coding layer, local feature modeling and semantic enhancement are first performed using residual blocks. Then, a downsampling module reduces the spatial resolution of the feature map and increases the number of feature channels. Simultaneously, an attention module is injected into the fourth coding layer to progressively improve the semantic abstraction level of the latent features. Specifically, latent features are extracted in the first coding layer of the UNet encoder. The second encoding layer extracts latent features. The third encoding layer extracts latent features. The fourth coding layer extracts latent features. .

5. The medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion according to claim 1, characterized in that, Medical images The global feature extraction network is input to extract global features. This includes the following steps: The convolution block module uses convolutional layers and linear layers sequentially... The k-th feature map is obtained by performing block embedding, projection, and flattening. ; The global feature extraction module The four-stage encoding process is performed step by step to obtain the results. , , , This constitutes a multi-scale global feature representation; specifically, each of the four feature extraction stages contains two consecutive SwingTransformer blocks, with the first stage directly... Process the data to output the first layer of global features. A downsampling module is set before the second, third, and fourth stages to halve the spatial size of the input feature map and double the number of channels. For the i-th stage (i=2,3,4): the output feature map of the (i-1)-th stage is... The data is sequentially processed by the downsampling module and two consecutive SwinTransformer blocks, and the global features extracted in the i-th stage are output. The feature representations are obtained sequentially. , , ; Each SwinTransformer block employs a structure that alternates between multi-head window attention and shifted window attention, with window_size set to 7. The downsampling module processes the input feature map (i=2,3,4) Process according to the following steps: Spatial resampling: Sampling is performed in the height and width dimensions of the feature map with a stride of 2, resulting in four dimensions. × × Sub-feature maps; Channel stitching: The four sub-feature maps mentioned above are stitched together along the channel dimension to obtain a size of... × × Intermediate features; Layer normalization: Perform layer normalization on the concatenated intermediate features; Linear projection: Transforming the number of channels from 4 using a linear transformation. Reduced to 2 This achieves a feature map spatial size that is halved, a channel number that is doubled, and output. in, Indicates the height of the sub-feature map, Indicates the width of the sub-feature map. This represents the number of channels in the sub-feature map.

6. The medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion according to claim 1, characterized in that, The global features Potential characteristics Input the medical image classification network to obtain fused features and predicted probability distribution This includes the following steps: In the first stage, the feature fusion module only uses global features. With latent features As input, the first-stage fused features are output through a window-based dual-stream cross-attention mechanism. ; In the second, third, and fourth stages, the feature fusion module of the i-th layer uses the global features of the i-th layer. Potential characteristics and the fusion features of the (i-1)th layer As input, Through cross-layer semantic guidance, dual-stream cross-attention, phased cascaded fusion, and selective channel reorganization, the following results are obtained sequentially. , , ; The classification module uses fourth-layer fusion features. As input, output image Predicted probability of belonging to each disease category , Where n represents the nth type of disease; the classification module includes a global average pooling module, a layer normalization module, and a fully connected classification module connected in sequence.

7. The medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion according to claim 6, characterized in that, The i-th layer feature fusion module uses the global features of the i-th layer. Potential characteristics and the fusion features of the (i-1)th layer As input, the process includes the following steps: cross-layer semantic guidance, dual-stream cross-attention, phased cascaded fusion, and selective channel reshaping. Fusion features of the (i-1)th layer Channel alignment and downsampling are performed to obtain a feature representation consistent with the feature space of the i-th layer. ,right Global average pooling and multilayer perceptron mapping are performed to generate cross-layer semantic guidance vectors. ; and exist Under the channel-level modulation, respectively, the following were obtained: , and to and Window-based two-stream cross-attention computation is performed to obtain the fused cross-modal features. ; Features Integration characteristics with the previous stage Concatenate along the channel dimension, and further concatenate with the original global features and latent features of the current layer in a multi-source manner to form a multi-source spliced ​​feature representation. ; Features Perform selective channel reshaping: for Perform a global average pooling operation in the spatial dimension and input it into a multilayer perceptron for nonlinear mapping to obtain... Then change the channel weight vector Acting on the original splicing features Obtain the weighted feature map ,right Nonlinear mapping can output ; The reshaped feature representation After channel alignment using 1×1 convolution, and... Perform residual connections to obtain the final fused features of the i-th layer. .

8. The medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion according to claim 7, characterized in that, The calculation of the dual-stream cross-attention of the window includes the following steps: Global features of the i-th layer and potential characteristics Layer normalization is performed separately to eliminate differences in the distribution of different features and stabilize feature representation; The normalized latent and global feature representations are divided into windows with a window_size of 7. Within each local window, a bidirectional cross-attention mechanism is constructed, which simultaneously "guides global features" and "reverses constraints latent features" to achieve symmetrical information interaction and structural alignment between different semantic spaces. Specifically, within each window, the attention response of latent features to global semantic information is calculated using latent features as the query vector and global features as the key and value vectors. At the same time, the attention response of global features to latent fine-grained structural information is calculated using global features as the query vector and latent features as the key and value vectors. The bidirectional cross-attention generates queries, keys, and values ​​through a learnable linear mapping, and independently calculates attention weights and performs weighted feature aggregation within each window to obtain enhanced latent feature representations and global feature representations, as shown in the following formulas: in Representing global features respectively Query, key, value They represent latent features respectively. The query, key, and value are T, where T represents the transpose operation of a matrix, and d represents the feature dimension of a vector. The attention results obtained in each window are restored to the original spatial layout through inverse window rearrangement, and residual connections are made with the corresponding input features to obtain the latent feature representation enhanced by cross-modal semantic interaction. , Finally, the two are concatenated along the channel dimension to form a block modal interaction feature representation. ;in , , These represent the number of channels, height, and width of the fused feature in the i-th layer, respectively.

9. The medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion according to claim 6, characterized in that, The classification module uses fourth-layer fusion features. As input, output image Predicted probability of belonging to each disease category This includes the following steps: The fourth layer of fusion features The input is fed into the global average pooling module, which performs an average operation on the spatial dimensions corresponding to each channel to obtain a channel-level global semantic feature representation. The global semantic feature representation is input into the layer normalization module to normalize the channel features of each sample; The features, after being processed by layer normalization, are input into a fully connected classification layer. The channel features are linearly mapped to the disease category space, and the image is output through a softmax activation function. Predicted probability distribution of each disease category .

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the medical image classification method based on LDM and Swin-Transformer hierarchical multi-scale feature fusion as described in any one of claims 1 to 9.

Images