Conditional diffusion medical image segmentation method fusing local and global features

Abstract

The invention discloses a conditional diffusion medical image segmentation method fusing local and global features. The method comprises the following steps: (1) acquiring and preprocessing a medical image sample; (2) extracting local texture and edge features of the image by using a convolutional neural network; (3) extracting global semantic and spatial dependency features of the image by using a network based on a self-attention mechanism; (4) fusing the local and global features, and generating condition feature representation for guiding a segmentation process; (5) inputting the conditional features into a conditional diffusion model, and generating a high-precision segmentation result in a step-by-step denoising mode; and (6) outputting a final medical image segmentation map. According to the method, local details and global semantic information are deeply fused through the diffusion model guided by the condition features, the problems of noise, fuzziness, complex structure and the like existing in the medical image are effectively solved, the segmentation accuracy and robustness are remarkably improved, and the method is suitable for automatic focus segmentation, anatomical structure quantitative analysis and clinical auxiliary diagnosis.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence, and in particular to a deep learning-based medical image segmentation method and system, which can be used for automatic segmentation of lesion areas and anatomical structures and to assist in clinical diagnosis. Background Technology

[0002] Medical image segmentation is a crucial research area in medical image processing and computer-aided diagnosis. Its goal is to automatically and accurately separate lesion regions or anatomical structures from medical images, providing technical support for disease diagnosis, treatment planning, and prognostic assessment. This technology can be widely applied in clinical scenarios such as skin lesion analysis, tumor delineation, and organ structure segmentation, and has been a research hotspot in the field of intelligent medical image analysis for many years. The design of the model framework is a core element in medical image segmentation technology; its structural rationality and feature representation ability directly affect the accuracy and stability of the segmentation results.

[0003] Currently, typical methods widely used in medical image segmentation are mainly based on deep learning model frameworks, such as Convolutional Neural Networks (CNNs) and their derived encoder-decoder structures. Related research shows that CNN-based segmentation models can effectively extract local texture and edge features from medical images, achieving certain results in various medical imaging tasks (see: Kayalibay, Baris, Grady Jensen, and Patrick Van Der Smagt. "CNN-based segmentation of medical imaging data." arXiv preprint arXiv:1701.03056 (2017).). However, due to the limitations of the local receptive field of convolutional operations, these methods still have shortcomings in modeling long-range spatial dependencies and global semantic information.

[0004] To overcome the aforementioned problems, Transformer models based on self-attention mechanisms have been increasingly introduced into the field of medical image segmentation in recent years. These methods enhance the model's ability to model long-distance dependencies through global attention, demonstrating certain advantages in segmenting complex structures (see: Xiao, Hanguang, et al. "Transformers in medical image segmentation: A review." Biomedical Signal Processing and Control 84 (2023): 104791.). However, Transformer models typically have limited ability to characterize local details and suffer from high computational complexity, making it difficult to balance segmentation accuracy and computational efficiency in high-resolution medical image scenarios.

[0005] In recent years, diffusion models, as a novel generative deep learning framework, have attracted widespread attention in tasks such as image generation and reconstruction. Some studies have attempted to introduce diffusion models into medical image segmentation tasks, modeling the segmentation results through a progressive denoising process to improve the overall consistency and robustness of the segmentation results (see: Wu, Junde, et al. "Medsegdiff: Medical image segmentation with diffusionprobabilistic model." Medical Imaging with Deep Learning. PMLR, 2024.). Furthermore, related patents have disclosed technical solutions for medical image segmentation using deep learning models (see patent: Xu Qian et al. Segmentation method of whole cerebral vascular system of zebrafish juveniles based on nnUNet trained on an independent dataset - application number / patent number: 202510985795.4).

[0006] While the aforementioned methods have improved medical image segmentation performance to some extent, existing technologies still have shortcomings: on the one hand, current deep learning models typically focus on modeling a single type of feature, making it difficult to simultaneously consider local detail features and global semantic information; on the other hand, the feature fusion methods between different model frameworks are relatively simple, limiting the overall expressive power of the model. Furthermore, existing diffusion-based medical image segmentation methods are mostly used as independent generation or post-processing modules, and have not yet achieved effective collaborative fusion with deep feature representations at the model framework level. Their conditional constraint mechanisms and local-global feature modeling capabilities still need further improvement.

[0007] In summary, existing medical image segmentation techniques still have room for improvement at the model framework level. There is an urgent need for a new deep learning model framework that, based on the fusion of local and global features, introduces a conditional modeling mechanism of diffusion model to improve the accuracy, stability, and structural consistency of medical image segmentation. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of existing medical image segmentation methods at the model framework level. Addressing the difficulty of existing deep learning models in simultaneously considering local detail features and global semantic information in medical images, and the lack of effective conditional constraints and feature coordination in diffusion models for medical image segmentation, this invention proposes a medical image segmentation method based on a deep learning model framework. This method improves the accuracy, stability, and structural consistency of medical image segmentation by integrating local feature extraction and global feature modeling mechanisms at the network structure level and introducing a conditional diffusion model to constrain and optimize the segmentation process.

[0009] The technical solution adopted in this invention is:

[0010] A medical image segmentation method based on a deep learning model framework, comprising the following steps:

[0011] Step 1: Acquire medical image samples and preprocess the medical image samples;

[0012] Step 2: Input the preprocessed medical image into the local feature extraction network to extract the local feature representation of the medical image;

[0013] Step 3: Input the preprocessed medical image into the global feature extraction network to extract the global feature representation of the medical image;

[0014] Step 4: Fuse the local feature representation with the global feature representation to generate a conditional feature representation to guide the segmentation process;

[0015] Step 5: Input the conditional feature representation into the conditional diffusion model, and generate the segmentation result of the medical image through stepwise denoising;

[0016] Step 6: Output the final medical image segmentation result.

[0017] in,

[0018] (1) Acquire medical image samples and preprocess the medical image samples;

[0019] Acquiring medical image samples The medical image samples are preprocessed, including image resizing and pixel value normalization, to obtain preprocessed medical images.

[0020] in, This indicates a preprocessing operation.

[0021] (2) Input the preprocessed medical image into the local feature extraction network to extract the local feature representation of the medical image;

[0022]

[0023] in, This represents a local feature extraction network. This represents the extracted local features, used to characterize the local texture and detailed structural information in medical images.

[0024] (3) Input the preprocessed medical image into the global feature extraction network to extract the global feature representation of the medical image;

[0025] Preprocessed medical images The input is a global feature extraction network, which uses a self-attention mechanism to perform global modeling of the medical image and obtain a global feature representation:

[0026]

[0027] in, This represents a global feature extraction network. This represents a feature representation that includes global contextual semantic information.

[0028] (4) The local feature representation and the global feature representation are fused to generate a conditional feature representation to guide the segmentation process;

[0029] Representing local features With global feature representation Perform feature fusion to generate conditional feature representations:

[0030] in, The conditional feature representation represents the feature fusion operation. It contains both local detail information and global semantic information, which are used to guide the subsequent segmentation generation process.

[0031] (5) Input the conditional feature representation into the conditional diffusion model, and generate the segmentation result of the medical image by stepwise denoising;

[0032] In conditional feature representation Guided by this model, a conditional diffusion model is constructed to model the segmentation results. During the forward diffusion process, noise is progressively added to the segmentation representation; during the backward diffusion process, denoising and reconstruction are performed based on the conditional feature representation. The denoising process is as follows:

[0033] in, Indicates the first The split state of the step, The parameter is A noise reduction network.

[0034] The final medical image segmentation result is obtained through a multi-step back diffusion process:

[0035] (6) Output the final medical image segmentation result.

[0036] Output medical image segmentation results generated by the conditional diffusion model. The segmentation results are used to characterize the spatial distribution of target regions in medical images.

[0037] Compared with the prior art, the advantages and effects of the present invention are as follows:

[0038] 1. This invention achieves automatic learning and expression of segmentation-related features by performing multi-level feature modeling on medical images, reducing reliance on manually designed features and empirical rules, enabling the model to more fully characterize the structural information in medical images, thereby improving the accuracy and stability of medical image segmentation results;

[0039] 2. This invention integrates local detail features and global semantic features of medical images and introduces a conditional constraint mechanism during the segmentation generation process. This allows the model to maintain the precision of the target boundary while ensuring overall structural consistency, effectively reducing missegmentation, boundary blurring, and structural discontinuities in the segmentation results, thus improving the overall quality of the segmentation results.

[0040] Other advantages of the present invention will be described below. Attached Figure Description

[0041] Figure 1 —Flowchart of this invention.

[0042] Figure 2 —Results of a medical image segmentation model.

[0043] Figure 3 —Visualization results of a medical image segmentation model. Detailed Implementation

[0044] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and detailed implementation methods and processes are given. However, the scope of protection of the present invention is not limited to the following embodiments.

[0045] I. Implementation Steps:

[0046] Step 1: Preprocess the medical image samples, including:

[0047] (1-1) The input medical image samples are medical images from the ISIC 2016 Skin Lesion Image Dataset, such as... Figure 2 The original image is shown in the figure. The ISIC 2016 dataset, publicly released by the International Skin Imaging Collaboration (ISIC), contains multiple clinically confirmed skin lesion image samples, primarily used for automated analysis and segmentation research of skin lesions. In this embodiment, the medical images in the ISIC 2016 dataset are color dermoscopic images. Each image typically contains three color channels: red, green, and blue, used to characterize the skin lesion area and its surrounding tissue structure. The original resolution of the medical images is not fixed, but the image size is generally in the high-resolution range to preserve key information such as lesion boundaries, texture structure, and color distribution. The ISIC 2016 dataset also provides manually annotated segmentation results corresponding one-to-one with the medical images, indicating the true spatial location of the skin lesion area, thus providing a standard reference for the training and validation of medical image segmentation methods.

[0048] (1-2) Preprocess each obtained medical image sample, including:

[0049] Each medical image in the ISIC 2016 dataset is resized to eliminate the impact of resolution differences between images on model training. In this embodiment, the original medical images are uniformly scaled to a preset size to meet the requirements of deep learning models for fixed input sizes. After resizing, the pixel values ​​of the medical images are normalized to map the image pixel values ​​to a predetermined range, thereby reducing the impact of brightness and contrast differences between different images on the model training process and improving the stability and convergence speed of model training. Through the above preprocessing operations, standardized medical image samples suitable for subsequent local feature extraction networks and global feature extraction networks are obtained while preserving as much as possible the boundary information of lesion regions, texture details, and color distribution features in the medical images.

[0050] Step 2: Input the preprocessed medical image samples into the local feature extraction network to extract local feature representations of the medical images, including:

[0051] The preprocessed medical image samples obtained in step 1 are input into a pre-constructed local feature extraction network. Through multi-layer convolution operations, the medical images are non-linearly mapped, thereby mapping the original image data to a high-dimensional feature representation space. The local feature extraction network uses the preprocessed medical images... As input, via parameters Local feature extraction function Obtain the corresponding local feature representation:

[0052]

[0053] in, The extracted local features are used to characterize the local texture features, edge information, and fine-grained spatial structure of lesion regions and anatomical structures in medical images. This represents the learnable parameters in the local feature extraction network. Through the aforementioned nonlinear mapping process, the original medical image samples are mapped from the pixel space to a high-dimensional feature space. In this feature space, the discriminability between different local structures is enhanced, providing a foundation for subsequent global feature modeling and feature fusion.

[0054] Step 3: Input the preprocessed medical image into the global feature extraction network to obtain the global feature representation of the medical image, including:

[0055] In the deep feature space, to characterize the long-range dependencies and global semantic associations between different spatial locations in medical images, a global feature extraction network based on a self-attention mechanism is introduced to perform global modeling of medical images.

[0056] Specifically, the preprocessed medical image obtained in step 1 is represented as a feature sequence:

[0057]

[0058] in, In medical images, the first Feature vectors corresponding to each spatial location Indicates the length of the feature sequence.

[0059] In the global feature extraction network, query vectors, key vectors, and value vectors are generated through linear mapping, and their calculation forms are as follows:

[0060] in, is a learnable mapping parameter matrix.

[0061] Based on the correlation between the query vector and the key vector, the attention weights between features are calculated as follows:

[0062] in, The weight matrix represents the global correlation between features. is the scaling factor for the feature dimension.

[0063] By performing a weighted summation of the value vectors, a global feature representation of the medical image is obtained:

[0064] in, This represents a feature representation that includes global contextual semantic information.

[0065] Due to attention weight It can adaptively characterize the correlation between features at different spatial locations. Therefore, when two spatial locations are semantically strongly related, their corresponding attention weights are larger; conversely, when the correlation is weak, their weights are smaller. Through this method, the global feature extraction network can effectively model long-range dependencies and overall structural information in medical images.

[0066] Step 4: Fuse the local feature representations and global feature representations to generate conditional feature representations to guide the segmentation process, including:

[0067] (4-1) Joint representation modeling of local and global features

[0068] In the deep feature space, the local features obtained in step 2 are represented

[0069] Compared with the global feature representation obtained in step 3

[0070] This is considered as a feature description of the same medical image obtained from different receptive fields and semantic levels.

[0071] To achieve collaborative modeling of the two, the local feature representation and the global feature representation are jointly represented by a feature fusion operator, and the fusion form is as follows:

[0072]

[0073] in, This represents the conditional feature representation after fusion. This represents a feature fusion operation, used to organically integrate local detail information and global semantic information in the feature space. In this embodiment, the feature fusion operation... This includes concatenating local and global features along the channel dimension and subsequent linear mapping to ensure consistency of the fused features at both the dimensional and semantic levels.

[0074] (4-2) Weighted modeling and optimized representation of conditional features

[0075] To further enhance the guiding ability of fused features for segmentation tasks, a learnable feature mapping function is introduced to perform weighted modeling on the fused feature representation, thereby obtaining the final conditional feature representation used for the conditional diffusion model.

[0076] Specifically, regarding fusion features The applied parameters are Feature mapping function Its calculation form is:

[0077]

[0078] in, This represents the conditional feature representation after weighted modeling, used as the conditional constraint input for subsequent conditional diffusion models. The feature mapping function... It can adaptively adjust the spatial location and channel information of the fused features, so that features that contribute more to the segmentation result receive higher responses, while the influence of redundant or irrelevant features is effectively suppressed. In this way, the fine-grained structural information contained in local features and the contextual semantic information contained in global features can be co-modeled in a unified feature space, thereby obtaining conditional feature representations with good discriminative and guiding capabilities;

[0079] Step 5: Input the conditional feature representation into the conditional diffusion model, and generate the segmentation result of the medical image through stepwise denoising, including:

[0080] Using the conditional features obtained in step 4

[0081] As a conditional constraint input, a conditional diffusion model is constructed to progressively generate and model the medical image segmentation results. During the forward diffusion process, Gaussian noise is progressively introduced into the target segmentation representation, causing it to gradually degenerate into a random noise distribution; during the backward diffusion process, the conditional feature representation... Guided by the parameter, The denoising network removes noise progressively, and its inverse denoising process is represented as follows:

[0082]

[0083] in, Indicates the first Segmentation representation under step diffusion state, This represents a conditional denoising network.

[0084] Through multi-step back-diffusion iteration, a medical image segmentation result with clear structure and continuous boundaries is gradually recovered from the initial noisy state:

[0085]

[0086] in, This represents the final medical image segmentation result.

[0087] Step 6: Output the medical image segmentation results generated by the conditional diffusion model, including:

[0088] Based on the inverse denoising generation result of the conditional diffusion model in step 5, the final stable and converged medical image segmentation representation is obtained:

[0089] The segmentation representation As the final segmentation result corresponding to the medical image under test, it is used to characterize the pixel-level spatial distribution of the target lesion region or anatomical structure in the medical image. The segmentation result can accurately reflect the morphological features and boundary information of the target region, and can be used for subsequent segmentation accuracy evaluation, result visualization, and auxiliary clinical diagnostic analysis.

[0090] II. The effects of the present invention are further illustrated by the following simulations:

[0091] 1. Simulation conditions and content:

[0092] To verify the effectiveness of the conditional diffusion medical image segmentation method based on local and global features proposed in this invention, simulation experiments were conducted on publicly available medical image datasets. This embodiment uses the ISIC 2016 skin lesion segmentation dataset as the experimental data source. This dataset, released by the International Skin Imaging Collaboration (ISIC), contains multiple clinically acquired and expert-annotated dermoscopic medical images and their corresponding lesion segmentation annotations, and is widely used for performance evaluation of medical image segmentation methods. In the experiment, the input medical images were RGB color dermoscopic images with inconsistent original resolutions. To meet the model input requirements, all medical images and their corresponding annotation masks in the dataset underwent uniform preprocessing operations, including image resizing and pixel value normalization, to obtain standardized model input data.

[0093] In the experimental setup, the dataset was divided into training and test sets. The training set was used for learning model parameters, and the test set was used to evaluate the model's segmentation performance. To ensure the stability and reliability of the experimental results, the data partitioning method and training process followed a unified experimental configuration. The simulation experiments were conducted on a deep learning-based computing platform, and the model training and inference processes were carried out in the same experimental environment to ensure the comparability and consistency of the experimental results.

[0094] 2. Simulation results:

[0095] Simulation results are attached. Figure 2 As shown. Figure 2 The performance comparison results of the proposed method with several existing representative medical image segmentation methods on the ISIC 2016 medical image segmentation dataset are presented. The evaluation metrics used include the Dice coefficient (DIC), Jaccard coefficient (JAC), and pixel-level classification accuracy (ACC). As shown in Table 1, the comparison methods include various typical medical image segmentation models such as FCN, U-Net, SegNet, TransUnet, CeNet, HRFB, DilatedSkinNet, DDN, and MedFuseNet. The experimental results in Table 1 show that the proposed method achieves the best results in all segmentation evaluation metrics. Specifically, the proposed method achieves a Dice coefficient of 0.952, a Jaccard coefficient of 0.912, and a pixel-level classification accuracy of 0.967, all of which are higher than the existing comparison methods.

[0096] Specifically, compared with traditional convolutional neural network-based segmentation methods such as FCN, U-Net, and SegNet, the method of this invention shows significant improvements in both Dice and Jaccard coefficients, indicating superior performance in overall lesion region segmentation accuracy and boundary consistency. Compared with TransUnet and CeNet methods that incorporate global modeling capabilities, the method of this invention further improves the segmentation accuracy of local detail regions while maintaining global semantic consistency. Furthermore, compared with existing segmentation methods based on diffusion models or multi-feature fusion structures (such as DDN and MedFuseNet), the method of this invention still has certain advantages in Dice and Jaccard coefficients, demonstrating that by introducing the organic fusion of local and global features at the model framework level and using conditional diffusion models to constrain the segmentation process, the stability and structural consistency of the segmentation results can be further improved. In summary, Figure 2The experimental results fully demonstrate that the conditional diffusion medical image segmentation method based on local and global features proposed in this invention has better overall performance in medical image segmentation tasks and can effectively improve segmentation accuracy and result reliability.

Claims

1. A conditional diffusion medical image segmentation method that integrates local and global features, comprising the following steps: Step 1: Acquire medical image samples and preprocess the medical image samples; Step 2: Input the preprocessed medical image into the local feature extraction network to extract the local feature representation of the medical image; Step 3: Input the preprocessed medical image into the global feature extraction network to extract the global feature representation of the medical image; Step 4: Fuse the local feature representation with the global feature representation to generate a conditional feature representation to guide the segmentation process; Step 5: Input the conditional feature representation into the conditional diffusion model, and generate the segmentation result of the medical image through stepwise denoising; Step 6: Output the final medical image segmentation result.

2. The conditional diffusion medical image segmentation method based on local and global features according to claim 1, characterized in that: Step 1, the preprocessing of the medical image samples includes the following steps: The acquired medical images are subjected to size unification processing and the image pixel values ​​are normalized to eliminate scale differences caused by different image acquisition conditions, thereby obtaining preprocessed medical image samples that meet the network input requirements.

3. The conditional diffusion medical image segmentation method based on local and global features according to claim 1, characterized in that: In step 2, the local feature extraction network is constructed based on a convolutional neural network structure and is used to extract local texture features and edge structure features from the preprocessed medical image. The obtained local feature representation is used to characterize the detailed information in the medical image.

4. The conditional diffusion medical image segmentation method based on local and global features according to claim 1, characterized in that: In step 3, the global feature extraction network is constructed based on a self-attention mechanism. By performing global modeling of medical image features, a global feature representation containing long-range spatial dependencies and global semantic information is obtained.

5. The conditional diffusion medical image segmentation method based on local and global features according to claim 1, characterized in that: In step 4, the local feature representation and the global feature representation are fused through a feature fusion operation, which includes at least one of feature concatenation, weighted fusion, or attention-guided fusion, to generate a conditional feature representation that simultaneously contains local detail information and global semantic information.

6. The conditional diffusion medical image segmentation method based on local and global features according to claim 1, characterized in that: In step 5, the conditional diffusion model includes a forward diffusion process and a reverse denoising process, wherein: During the forward diffusion process, random noise is gradually introduced into the segmented representation; In the reverse denoising process, guided by the conditional feature representation, noise is gradually removed through a parameterized denoising network to reconstruct the segmentation result of the medical image.

7. The conditional diffusion medical image segmentation method based on local and global features according to claim 6, characterized in that: The reverse denoising process can be expressed as: in, Indicates the first The split state of the step, This represents the conditional feature representation. The parameter is A noise reduction network.

8. The conditional diffusion medical image segmentation method based on local and global features according to claim 1, characterized in that: In step 6, the output medical image segmentation results are used to characterize the spatial distribution of target lesion regions or anatomical structures in medical images, and can be used to assist in clinical diagnosis or medical image analysis.

Images