A medical image segmentation method based on region-boundary mutual learning diffusion model

Abstract

The application discloses a medical image segmentation method based on a region-boundary mutual learning diffusion model, which comprises the following steps: 1, constructing a network model RBML-Diff and initializing hyperparameters; 2, preprocessing unlabeled medical images, labeled medical images and labeled medical image true values; 3, in the process of forward diffusion of the network model, adding noise to the labeled medical image true values to obtain a noise mask image; 4, inputting the noise mask image and the unlabeled medical image into a conditional encoder to output a feature map; 5, performing region perception and feature refinement on the feature map output by the conditional encoder by a layer-in feature interaction module and a layer-inter feature cooperation module in a region perception module; 6, extracting the boundary of the unlabeled medical image by using a Laplace operator, repeatedly passing through a region-boundary mutual learning module and an upsampling operation to obtain a prediction result; and 7, gradually performing reverse diffusion denoising on the prediction result to obtain a segmentation mask image. The application effectively solves the problems of large-scale changes and boundary ambiguity in medical image segmentation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image segmentation, and specifically relates to a medical image segmentation method based on a region-boundary mutual learning diffusion model. BACKGROUND

[0002] Medical image segmentation technology plays a crucial role in the field of clinical medicine. It can extract target regions or tissues in images, providing key evidence for doctors to analyze pathological features and helping doctors make more accurate diagnoses.

[0003] Traditional medical image analysis relies on the experience of doctors and manual segmentation, which is not only time-consuming and labor-intensive but also easily affected by subjective factors. With the advancement of medical image segmentation technology, automatic segmentation has become a research hotspot. However, due to the low contrast of medical images, the diversity of target region shapes and sizes, and the ambiguity of the boundaries between target regions and surrounding normal tissues, automatic medical image segmentation remains a highly challenging task.

[0004] With the development of deep learning technology, researchers have proposed various automatic medical image segmentation models, mainly divided into convolutional neural network (CNN) based and Transformer based segmentation models. CNN based segmentation models often fail to fully capture the relationships and dependencies between different regions in medical images, lacking a deep understanding of the global image. Transformer based segmentation models are good at capturing global information, enhancing the network's understanding of global context, but tend to overlook local details, underperforming in identifying small lesion regions and handling fuzzy boundaries.

[0005] Diffusion models have excellent feature learning capabilities in the process of reverse denoising and have received widespread attention. Unlike CNN and Transformer, diffusion models model the evolution process of target regions as a parameterized Markov chain, helping to learn the statistical distribution characteristics of target regions and thus more accurately segment images. Some scholars have applied diffusion models to image segmentation tasks and achieved good segmentation results. For example: Amit et al. first applied diffusion models to image segmentation problems and introduced the concept of multiple generations, effectively improving diffusion models and enhancing image segmentation performance; Wu et al. first integrated Transformer into diffusion models for medical image segmentation and proposed SSFormer to simulate the interaction between noise and semantic features; Chen et al. proposed a conditional Bernoulli diffusion model, which uses Bernoulli noise as the diffusion kernel instead of Gaussian noise, effectively enhancing the model's ability to handle binary segmentation tasks and achieving faster sampling speed.

[0006] Although the diffusion model has great potential and application value in image segmentation tasks, there are still four problems when applying the diffusion model to medical image segmentation tasks: (1) The complexity of different sizes of lesions cannot be fully considered when extracting conditional features, resulting in poor segmentation performance for lesions of different sizes and relatively hidden lesion regions; (2) The influence of feature diversity on segmentation performance is ignored when extracting conditional features from the original image or noise mask; (3) Different levels of features cannot be integrated according to the diversity of semantic features, resulting in the inability to fully capture the key semantic information of the lesion region boundary and low segmentation accuracy; (4) For the problem of difficult-to-distinguish boundaries, the existing diffusion model cannot effectively integrate features, resulting in poor segmentation effect of the boundary fuzzy lesion. SUMMARY

[0007] In view of the defects of the prior art, the purpose of the present application is to provide a medical image segmentation method based on a region-boundary mutual learning diffusion model, which can accurately identify lesions of various shapes, sizes and boundary blur, and improve the segmentation accuracy of medical images.

[0008] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0009] A medical image segmentation method based on a region-boundary mutual learning diffusion model, comprising the following steps:

[0010] Step 1, constructing a network model RBML-Diff and initializing hyperparameters;

[0011] The network model RBML-Diff includes a conditional encoder, a region perception module and a boundary perception denoising network, wherein: the region perception module includes an intra-layer feature interaction module and an inter-layer feature collaboration module; the boundary perception denoising network includes a segmentation encoder and a region-boundary mutual learning module;

[0012] Step 2, preprocessing the unlabeled medical image to be segmented , the labeled medical image and the labeled medical image ground truth ;

[0013] Step 3, inputting the preprocessed labeled medical image and the labeled medical image ground truth into the network model RBML-Diff, using the forward diffusion process of the network model RBML-Diff to perform noise addition processing on , and after step iterations, obtaining a noise mask image after noise addition;

[0014] Step 4, inputting the noise mask image and the preprocessed unlabeled medical image to be segmented The input condition encoder performs feature extraction to obtain a feature map 、 、 and ;

[0015] Step 5, inputting the 、 、 and respectively into different intra-layer feature interaction modules in the region perception module, the intra-layer feature interaction module performs feature mining on the feature map 、 、 and to obtain a feature map 、 、 and , inputting the feature map , and , respectively into two inter-layer feature collaboration modules of the region perception module, the inter-layer feature collaboration module performs cross-layer feature refinement on the feature map , and , to obtain a refined multi-scale region feature map and ;

[0016] Step 6, extracting the boundary of the preprocessed unlabeled medical image to be segmented by a Laplacian operator to obtain a boundary feature map , and extracting the features of the noise mask map by using the segmentation encoder in the boundary perception denoising network , splicing the output feature map of the segmentation encoder and the feature map , and performing upsampling to obtain a predicted feature map ;

[0017] Setting the predicted feature map output by the region-boundary mutual learning module in the boundary perception denoising network as , first performing Inward predicted feature map Respectively, the opposite attention map and the boundary attention map are obtained by taking the opposite and extracting the boundary through the Laplacian operator And the boundary attention map The region feature 、 And Respectively, the boundary feature map , the next level of the opposite attention map And the boundary attention map Cross learning, refining in channels and space, and upsampling to obtain , repeating the region-boundary mutual learning module and upsampling operation, and finally obtaining the prediction result ;

[0018] Step 7, the prediction result is gradually diffused and denoised to obtain the final segmentation mask map .

[0019] Further, the initialization hyperparameters of step 1 include setting the training batch size to 32, the training iteration number to 150, and using a learning rate decay strategy during training, with the initial learning rate set to 0.0001.

[0020] Further, the specific process of step 2 is:

[0021] First, the unlabeled medical image to be segmented is data enhanced, and then the enhanced unlabeled medical image to be segmented is normalized.

[0022] The size of the labeled medical image and the labeled medical image ground truth is adjusted to 256x256.

[0023] Further, the data enhancement includes random scaling, horizontal and vertical flipping, random angle rotation, random noise filling, and random cropping.

[0024] Further, the specific process of step 3 is:

[0025] In the forward diffusion process of the network model RBML-Diff, by continuously Step iteration, Gaussian noise is gradually added to the original labeled medical image ground truth , and the forward diffusion process is represented as:

[0026]

[0027] wherein, represents the number of diffusion steps, and respectively represent the noise mask map at time and respectively represent the noise mask map at time represents the noise mask map at time 1, 2, …, T

[0028] In each iteration process, the formula of adding Gaussian noise is represented as:

[0029]

[0030] wherein, is a parameter for controlling Gaussian noise in the forward diffusion process, is a unit matrix of is a Gaussian distribution; The process of sampling at any time

[0031]

[0032]

[0033]

[0034]

[0035] wherein, is an attenuation coefficient related to represents the product of all in the forward diffusion process. Further, the specific process of step 4 is:

[0036] Step 4.1, taking the noise mask map

[0037] and the preprocessed unlabeled medical image to be segmented as the input of the conditional encoder, the conditional encoder includes four stages, the first stage includes a fusion embedding layer and a Transformer encoder, the Transformer encoder of the first stage outputs a feature map ;

[0038] Step 4.2, taking the feature map as the input of the second stage of the conditional encoder, the Transformer encoder of the second stage outputs a feature map ;

[0039] Step 4.3, taking the feature map​​​​ Input condition encoder third stage, the third stage of the transformer encoder output feature map

[0040] Step 4.4, the feature map Input condition encoder fourth stage, the fourth stage of the transformer encoder output feature map

[0041] Further, the specific process of the step 5 is:

[0042] Step 5.1, respectively, the feature map 、 、 And Input into the four layer feature interaction modules of the region perception module, realize different direction and scale feature interaction, the four layer feature interaction modules output feature map , 、 And , denoted as , expressed as:

[0043]

[0044]

[0045] In the formula, The first feature map generated by the backbone network is represented by , the number of branches is represented by , the output of the first branch is represented by , the element addition is represented by , the asymmetric convolution is represented by , the convolution is represented by , the convolution is represented by , the three branch splicing is represented by , and the activation function is represented by ReLU

[0046] Step 5.2, the feature map , ) and , are respectively input into the two inter-layer feature coordination modules of the region perception module to strengthen the shared feature expression, and the output feature maps and​​​​​ , is represented as:

[0047]

[0048]

[0049]

[0050] In the formula, represents two ordinary convolutions, represents a hyperbolic tangent function, represents norm, represents element multiplication, represents matrix transposition, represents feature maps after two 1x1 convolution processing;

[0051] Further, the specific process of step 6 is:

[0052] Step 6.1, extracting the boundary features of the preprocessed unlabeled medical image to be segmented by a Laplacian operator to obtain a Laplacian boundary feature map , is represented as:

[0053]

[0054]

[0055]

[0056]

[0057] In the formula, represents an input preprocessed unlabeled medical image to be segmented, represents a convolution operator with a Gaussian filter, represents down-sampling, represents up-sampling, represents a Laplacian image of the Laplacian pyramid k layer, represents a feature map of the preprocessed unlabeled medical image to be segmented after Gaussian convolution and down-sampling;

[0058] Step 6.2, inputting the noise mask map into the segmentation encoder of the boundary-aware denoising network, and then splicing the output feature map of the segmentation encoder and the feature map , and performing up-sampling to obtain a predicted feature map ;​

[0059] Step 6.3, set the predicted feature map output by the region-boundary mutual learning module in the boundary-aware denoising network as , and the predicted feature map is taken as the input of the region-boundary mutual learning module. Take the inverse, highlight the non-significant region, and get the reverse attention map At the same time, the Laplace operator is used to extract the boundary of the predicted feature map , and the boundary attention map is obtained. The reverse attention map and the boundary attention map are represented as:

[0060]

[0061]

[0062] In the formula, represents the Laplace operator;

[0063] Step 6.4, cross learning the region feature map with the Laplace boundary feature map , the reverse attention map of the next level and the boundary attention map respectively through the region-boundary mutual learning module, represented as:

[0064]

[0065]

[0066] In the formula, represents the Sigmoid activation function;

[0067] Step 6.5, refine the output feature map in the channel and space, represented as:

[0068]

[0069] In the formula, represents the channel attention operation, represents the spatial attention operation;

[0070] Step 6.6, upsample to get the result , repeat the region-boundary mutual learning module and the upsampling operation to get the predicted result .

[0071] Further, the specific process of reverse diffusion denoising in step 7 is represented as:

[0072]

[0073]

[0074]

[0075] In the formula, And Respectively represent the noise mask map at the moment And The moment, Indicate the mean value predicted by the network model under the condition of the moment And the current noise mask map , Indicate the variance of the Gaussian distribution in the reverse diffusion process, Is a parameter for controlling Gaussian noise in the reverse diffusion process, Is the attenuation coefficient related to , Indicate the product of all In the reverse diffusion process, Is a segmentation mask.

[0076] Compared with the prior art, the present application has the following technical effects:

[0077] The network model RBML-Diff proposed by the present application includes a region perception module and a boundary perception denoising network, wherein: the region perception module includes an intra-layer feature interaction module and an inter-layer feature cooperation module, the intra-layer feature interaction module focuses on the diversity within single-layer characteristics, and the target of the inter-layer feature cooperation module is to fully utilize the potential of multi-layer characteristics, thereby facilitating the region perception module to deeply mine and perfect multi-scale region characteristics, significantly improving the recognition ability of the region perception module to the diversity of the segmentation target form, reducing the influence of feature diversity on the segmentation performance, and being more suitable for segmenting complex lesion images of different sizes and forms; the boundary perception denoising network precisely captures boundary information by using a Laplace morphological operator, fuses multi-scale region characteristics and high-frequency boundary characteristics by using a region-boundary mutual learning module, and through such mutual learning, the RBML-Diff fully utilizes multi-level features from regions and boundaries in the image to capture key semantic information of the lesion region boundary, and further generates an accurate segmentation image result. In short, the present application effectively solves the problems of large-scale change and boundary ambiguity in medical image segmentation, can accurately extract the contour of the to-be-segmented target of the medical image, provides technical support for early clinical diagnosis and treatment, and has a broad application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1is a structural schematic diagram of a network model RBML-Diff of the present application;

[0079] Figure 2 is a structural schematic diagram of an intra-layer feature interaction module of the present application;

[0080] Figure 3 is a structural schematic diagram of an inter-layer feature coordination module of the present application;

[0081] Figure 4 is a structural schematic diagram of a region-boundary mutual learning module of the present application;

[0082] Figure 5 is a visualization result of the network model RBML-Diff of the present application and an existing network on a polyp segmentation data set. DETAILED DESCRIPTION

[0083] The specific content of the present application is further explained in detail in the following combined with embodiments.

[0084] A medical image segmentation method based on a region-boundary mutual learning diffusion model, comprising the following steps:

[0085] Step 1, constructing a network model RBML-Diff as shown in Figure 1 , initializing hyperparameters;

[0086] The network model RBML-Diff comprises a conditional encoder, a region perception module and a boundary perception denoising network, wherein the conditional encoder comprises four stages, the first stage comprises a fusion embedding layer and a Transformer encoder, and the second stage, the third stage and the fourth stage each comprise an overlapping embedding layer and a Transformer encoder; the region perception module comprises four intra-layer feature interaction modules as shown in Figure 2 and two inter-layer feature coordination modules as shown in Figure 3 ; the boundary perception denoising network comprises a segmentation encoder and three region-boundary mutual learning modules as shown in Figure 4 ;

[0087] The initialization of the hyperparameters comprises: setting the training batch size to 32, setting the training iteration number to 150, and in the training process, adopting a learning rate decay strategy, and setting the initial learning rate to 0.0001;

[0088] Step 2, preprocessing the unlabeled medical image to be segmented, the labeled medical image and the labeled medical image ground truth , the specific process being:

[0089] the unlabeled medical image Data augmentation is performed, including random scaling, horizontal and vertical flipping, random selection of angle for rotation, random filling of noise and random cropping, and then normalization processing is performed;

[0090] The labeled image and the true value of the labeled image are resized to 256x256;

[0091] Step 3, input the preprocessed labeled medical image and the labeled medical image true value into the network model RBML-Diff, use the forward diffusion process of the network model RBML-Diff to add noise to , and after step iterations, obtain the noise mask image after noise addition, the specific process is as follows:

[0092] In the forward diffusion process of the network model RBML-Diff, by continuously iterating steps, Gaussian noise is gradually added to the original labeled medical image true value , and the forward diffusion process is represented as:

[0093]

[0094] In the formula, represents the diffusion step number, and represent the noise mask image at time and time in the forward diffusion process, represents the noise mask image at 1,2,..., T in the diffusion process;

[0095] In each iteration process, the formula for adding Gaussian noise is:

[0096]

[0097] In the formula, is a parameter for controlling Gaussian noise in the forward diffusion process, is a unit matrix of , and is a Gaussian distribution;

[0098] The process of sampling at any time is represented as:

[0099]

[0100]

[0101]

[0102] In the formula, Is with The relevant attenuation coefficient, Indicates all during the forward diffusion process The product;

[0103] Step 4: Obtain the noise mask image And preprocessed unlabeled medical images to be segmented The input conditional encoder is used for feature extraction to obtain feature maps. 、 、 and The specific process is as follows:

[0104] Step 4.1: Obtain the noise mask image. And preprocessed unlabeled medical images to be segmented The feature map is used as input to the conditional encoder and output by the first-stage Transformer encoder. ;

[0105] Step 4.2: Transfer the feature map The second stage of the input conditional encoder, and the output feature map of the second-stage Transformer encoder. ;

[0106] Step 4.3: Transfer the feature map The third stage of the input conditional encoder, and the output feature map of the Transformer encoder in the third stage. ;

[0107] Step 4.4: Transfer the feature map The fourth stage of the input conditional encoder, and the output feature map of the fourth stage Transformer encoder. ;

[0108] Step 5, 、 、 and The four inputs to the region perception module are as follows: Figure 2 The intra-layer feature interaction module shown above interacts with features. 、 、 and Feature mining is performed at different directions and scales to obtain feature maps. 、 、 and , feature map ( , One of the input region sensing modules, such as Figure 3 The inter-layer feature collaboration module shown will ( , The input to another area-aware module is as follows: Figure 3 The inter-layer feature collaboration module shown has two modules that respectively process the feature map ( , )and( , Cross-layer feature refinement is performed to obtain refined multi-scale region feature maps. and The specific process is as follows:

[0109] Step 5.1: Separate the feature maps 、 、 and The input is fed into four intra-layer feature interaction modules of the region perception module to achieve feature interaction at different directions and scales. The four intra-layer feature interaction modules output feature maps respectively. , 、 and , recorded as , is represented as:

[0110]

[0111]

[0112] In the formula, The first generation representing the backbone network Each feature map Represents the number of branches. Representing the The output of each branch Represents element addition. Represents asymmetric convolution. represent convolution, represent convolution, Representatives such asFigure 2 The diagram shows the concatenation of the three branches of the intra-layer feature interaction module. yes ReLU Activation function;

[0113] Step 5.2, transfer the feature map ( , )and( , The features are respectively input into the two inter-layer feature collaboration modules of the region perception module to enhance the shared feature representation and obtain the output feature map. and , represented as:

[0114]

[0115]

[0116]

[0117] In the formula, Represents two ordinary convolutions, Represents the hyperbolic tangent function. represent Norm, Represents element-wise multiplication. Represents matrix transpose. represent Feature map after two 1×1 convolutions;

[0118] Step 6: Extract the preprocessed, unlabeled medical image to be segmented using the Laplacian operator. The boundary is obtained, and the boundary feature map is obtained. 、 and , recorded as Then use such Figure 4 The region-boundary mutual learning module shown learns from and fuses the features of the region and boundary. The specific process is as follows:

[0119] Step 6.1: Extract the preprocessed, unlabeled medical image to be segmented using the Laplacian operator. The boundary features are used to obtain the Laplace boundary feature map. , represented as:

[0120]

[0121]

[0122]

[0123]

[0124] wherein, represents an input pre-processed unlabeled medical image to be segmented, represents a convolution operator with a Gaussian filter, represents down-sampling, represents up-sampling, represents a Laplacian image of the k layer of the Laplacian pyramid, represents a pre-processed unlabeled medical image to be segmented after Gaussian convolution and down-sampling;

[0125] Step 6.2, the noise mask map is input into the segmentation encoder of the boundary-aware denoising network, and the output feature map of the segmentation encoder and the feature map are spliced and up-sampled to obtain a predicted feature map ;

[0126] Step 6.3, the predicted feature map output by the region-boundary mutual learning module in the boundary-aware denoising network is set as , the predicted feature map is taken inversely, and the insignificant region is highlighted to obtain an inverse attention map , and meanwhile, the boundary of the predicted feature map is extracted by using a Laplacian operator to obtain a boundary attention map , the inverse attention map and the boundary attention map are represented as:

[0127]

[0128]

[0129] wherein, represents a Laplacian operator;

[0130] Step 6.4, the region feature map is cross-learned with the Laplacian boundary feature map , the inverse attention map of the next level and the boundary attention map of the next level by the region-boundary mutual learning module, and is represented as:

[0131]

[0132] ​

[0133] In the formula, This represents the Sigmoid activation function;

[0134] Step 6.5: Process the output feature map Further refinement within the passageway and space is represented as follows:

[0135]

[0136] In the formula, Represents channel attention operations. Represents spatial attention operations;

[0137] Step 6.6, for The result was obtained by upsampling. The prediction result is finally obtained by repeatedly using the region-boundary mutual learning module and upsampling operation. ;

[0138] Step 7: Analyze the prediction results. By progressively reverse diffusion denoising, the final segmentation mask image is obtained. The process of reverse diffusion denoising is represented as:

[0139]

[0140]

[0141]

[0142] In the formula, and These represent the reverse diffusion process. and Noise mask at time step, Indicates in Time and current noise mask image Under the given conditions, the mean predicted by the model, This represents the variance of the Gaussian distribution during the backdiffusion process. These are the parameters used to control Gaussian noise during the back diffusion process. Is with The relevant attenuation coefficient, Indicates all during the reverse diffusion process The product of This is a segmentation mask image.

[0143] In order to verify the performance of the network model RBML-Diff proposed in the embodiment in the medical image segmentation process, the polyp images in the existing public data sets Kvasir, CVC-ClinicDB, CVC-300, CVC-ColonDB and ETIS-LaribPolypDB are used for network training and testing. The training data set has a total of 1450 images, of which 900 images are from Kvasir and 550 images are from CVC-ClinicDB; 100 images are randomly selected from the training set as a validation set for verifying the performance of the network model RBML-Diff in the medical segmentation process. Some images are selected from the five data sets for testing, of which: 100 images are selected from Kvasir, 62 images are selected from CVC-ClinicDB, 60 images are selected from CVC-300, 380 images are selected from CVC-ColonDB, and 196 images are selected from ETIS-LaribPolypDB, and the test data are not used for training. In the training process, the AdamW optimizer is used to update the parameters of the network model RBML-Diff.

[0144] In order to test the accuracy and superiority of the image segmentation method proposed in the embodiment, the following experiments are further illustrated, wherein:

[0145] 1) The verification environment is: the CPU is Intel(R) Xeon(R) Gold 6226R; the memory is 32GB; the GPU is Nvidia Geforce RTX 3090 with 24GB of video memory, and the operation is carried out on the Ubuntu 16.04.10 operating system. The deep learning framework used by the network model RBML-Diff of the embodiment is PyTorch;

[0146] 2) The segmentation performance is evaluated by the following index parameters, which are:

[0147]

[0148]

[0149]

[0150]

[0151]

[0152]

[0153]

[0154] In the formula, , and respectively represent the number of true, true negative and false negative samples, represents the precision, represents the recall, refers to a set similarity measure function, which is usually used to calculate the similarity of two samples; refers to the ratio of the predicted target and the real label, which represents the similarity or overlap of two samples; is the weighted average of precision and recall, is used to evaluate the structural similarity between the predicted segmentation result and the real segmentation, is the average structural similarity over the entire image, is the maximum structural similarity over the entire image, refers to the mean absolute error, which is used to measure the difference between the real value and the predicted value, , , , , and The value range of is [0, 1], the closer the value is to 1, the better the segmentation effect is, the closer the value is to 0, the better the segmentation effect is.

[0155] Firstly, the ablation experiment is used to verify the role of the intra-layer feature interaction module, the inter-layer feature coordination module and the region-boundary mutual learning module in the RBML-Diff proposed in the embodiment. The RBML-Diff and the pyramid Transformer-based Baseline using the same architecture are compared on five data sets. Specifically, the three modules are removed from the RBML-Diff respectively, and compared with the Baseline to verify the effectiveness of the three modules by counting mDice and mIoU. The ablation experiment results are shown in Table 1.

[0156] Table 1 Ablation experiment results of modules of RBML-Diff

[0157]

[0158] As shown in Table 1, compared with the RBML-Diff, after removing the intra-layer feature interaction module, the mDice and mIoU scores on the data sets Kvasir, CVC-ClinicDB, CVC-ColonDB and ETIS-LaribPolypDB all decrease, especially on the ETIS-LaribPolypDB data set, and scores decrease, especially on the ETIS-LaribPolypDB data set, The performance decreased by 2.3%, but it performed best on the CVC-300 dataset. Observing the images in the CVC-300 dataset, we can see that the polyp images are similar in size and very regular, and the dataset size is relatively small. This is precisely why better performance can be achieved without the intra-layer feature interaction module. Similarly, removing the inter-layer feature collaboration module improves performance on all datasets. and The scores all decreased, especially in CVC-ClinicDB. The score dropped by 1.6% in ETISLaribPolypDB. The score decreased by 1.4%, with relatively smaller decreases on other datasets; furthermore, without the region-boundary mutual learning module, the score on all five datasets... and The more significant decrease in scores demonstrates the importance of the region-boundary mutual learning module in improving model performance. Compared with the intra-layer feature interaction module and the inter-layer feature collaboration module, the region-boundary mutual learning module has a more significant impact on model performance.

[0159] Tables 2 through 6 show the segmentation results of RBML-Diff and mainstream segmentation network models in recent years on five publicly available polyp datasets. The best results are indicated in bold red, and the second-best results are indicated in bold blue. The specific results are as follows:

[0160] Table 2. Quantitative results of different network models on the Kvasir dataset.

[0161]

[0162] As can be seen from Table 2, on the Kvasir dataset, RBML-Diff not only in and They achieved excellent scores of 0.925 and 0.879, and still... 、 、 、 and The highest evaluation scores were achieved, specifically, compared to existing models Polyp-PVT and CTNet, the network model RBML-Diff proposed in this embodiment achieved the highest scores. Both improved by 0.8%, compared to the existing model Polyp-PVT, RBML-Diff... An increase of 1.5%;

[0163] Table 3 Quantitative results of different network models on the CVC-ClinicDB dataset

[0164]

[0165] As shown in Table 3, on the CVC-ClinicDB dataset, RBML-Diff achieved the highest evaluation scores in , , and , and also outperformed most existing network models in other metrics;

[0166] Table 4 Quantitative results of different network models on the CVC-300 dataset

[0167]

[0168] As shown in Table 4, on the CVC-300 dataset, RBML-Diff achieved scores of 0.910, 0.847, 0.898, 0.945, 0.976 and 0.005 in , , , , and , respectively, all of which were higher than those of other network models, and also achieved a sub-optimal score of 0.982;

[0169] Table 5 Quantitative results of different network models on the CVC-ColonDB dataset

[0170]

[0171] As shown in Table 5, on the CVC-ColonDB dataset, RBML-Diff outperformed all existing network models in all evaluation metrics, and compared with CTNet which ranked second, it achieved significant improvements of 2.4% and 2.7% in and , respectively; meanwhile, , , , and were also improved by 1.8%, 1.3%, 1%, 1.1% and 0.2%, respectively;

[0172] Table 6 Quantitative results of different network models on the ETIS-LaribPolypDB dataset

[0173]

[0174] As can be seen from Table 6, on the ETIS-LaribPolypDB dataset, although the evaluation index score of RBML-Diff is lower than that of model CTNet, the performance of RBML-Diff is very close to that of CTNet, and the gap is related to the multiple target polyp images contained in the ETIS-LaribPolypDB dataset;

[0175] It can be known from the analysis of the results in Tables 2-6 that, compared with other existing models, RBML-Diff exhibits good learning ability and generalization ability on different datasets.

[0176] Figure 5 The visualization results of the network model RBML-Diff proposed in the embodiment and the existing network model on the polyp segmentation dataset are shown. It can be seen that RBML-Diff always has robust segmentation ability on polyps of different sizes and types, and the adaptability and accuracy are better than those of other network models. In addition, RBML-Diff has superior perception performance on the fuzzy edge of the polyp, and can more accurately segment the polyp target boundary.

Claims

1. A medical image segmentation method based on a region-boundary mutual learning diffusion model, characterized in that, The method comprises the following steps: Step 1, constructing a network model RBML-Diff and initializing hyperparameters; The network model RBML-Diff comprises a conditional encoder, a region perception module and a boundary perception denoising network, wherein: the conditional encoder comprises four stages, the first stage comprises a fusion embedding layer and a Transformer encoder, the second stage, the third stage and the fourth stage each comprise an overlapping embedding layer and a Transformer encoder; the region perception module comprises an intra-layer feature interaction module and an inter-layer feature coordination module; the boundary perception denoising network comprises a segmentation encoder and a region-boundary mutual learning module; Step 2, pre-processing the unlabeled medical images to be segmented , the labeled medical images and the labeled medical image ground truth ​ Step 3, pre-processing the labeled medical image and the labeled medical image ground truth inputting the network model RBML-Diff, performing noise adding processing on the by using the forward diffusion process of the network model RBML-Diff, and obtaining a noise mask image after noise adding processing by step iteration ; Step 4, generating a noise mask map and the preprocessed unlabeled medical image to be segmented inputting the conditional encoder to perform feature extraction to obtain a feature map 、 、 and ; Step 5, inputting the feature map (F1) into the region perception module, and inputting the feature map (F2) into the region perception module, wherein the region perception module comprises an intra-layer feature interaction module and an inter-layer feature cooperation module. 、 、 and respectively into different intra-layer feature interaction modules in the region perception module, and the intra-layer feature interaction module performs feature interaction on the feature map (F1) and the feature map (F2) to obtain a feature map (F3) and a feature map (F4). 、 、 and perform feature mining in different directions and scales to obtain a feature map (F5) and a feature map (F6). 、 、 and , and inputting the feature map (F5) and the feature map (F6) into two inter-layer feature cooperation modules of the region perception module, and the inter-layer feature cooperation module performs cross-layer feature refinement on the feature map (F5) and the feature map (F6) to obtain a refined multi-scale region feature map (F7) and a refined multi-scale region feature map (F8). , , , , and ;​​​​ Step 6: Extract the preprocessed, unlabeled medical image to be segmented using the Laplacian operator. The boundary is obtained, and the boundary feature map is obtained. The noise mask image is extracted using the segmentation encoder in the boundary-aware denoising network. The features will be used to segment the encoder's output feature map and feature map. After concatenation and upsampling, the predicted feature map is obtained. ; The prediction feature map output by the region-boundary mutual learning module in the boundary-aware denoising network is set as , the prediction feature map including is respectively taken to the opposite and the boundary is extracted by a Laplacian operator to obtain an inverse attention map and a boundary attention map , and then the region feature 、 and are respectively cross-learned with the boundary feature map , the inverse attention map of the next level and the boundary attention map , are refined in channels and spaces, and then are up-sampled to obtain , the region-boundary mutual learning module and the up-sampling operation are repeated, and finally the prediction result is obtained. Step 7, prediction results Stepwise backward diffusion denoising to obtain the final segmentation mask map .

2. The medical image segmentation method based on the region-border mutual learning diffusion model according to claim 1, characterized in that, The initialization of the hyperparameters in step 1 comprises: setting the training batch size to 32, setting the training iteration number to 150, and in the training process, adopting a learning rate decay strategy, and setting the initial learning rate to 0.0001. 3.The medical image segmentation method based on the region-border mutual learning diffusion model according to claim 1, characterized in that, The specific process of step 2 is: First, unlabeled medical images to be segmented Data augmentation is performed on the augmented, unlabeled medical images to be segmented. Perform normalization processing; annotated medical images and annotated medical image ground truths are resized to 256x256.

4. The medical image segmentation method based on the region-border mutual learning diffusion model according to claim 3, characterized in that, The data enhancement comprises random scaling, horizontal and vertical flipping, randomly selecting an angle for rotation, randomly filling noise and random cropping. 5.The medical image segmentation method based on the region-border mutual learning diffusion model according to claim 1, characterized in that, The specific process of step 3 is: In the forward diffusion process of the network model RBML-Diff, the Gaussian noise is gradually added to the original labeled medical image true value through continuous step iterations, and the forward diffusion process is represented as: step iterations, and the forward diffusion process is represented as: wherein represents the number of diffusion steps, and respectively represent the noise mask map at the time instant and respectively represent the noise mask map at the time instant represents the noise mask map at the time instant 1, 2,... T respectively represent the noise mask map at the time instant In each iteration process, the formula for adding Gaussian noise is: wherein is a parameter controlling the Gaussian noise in the forward diffusion process, is a is the identity matrix, is a Gaussian distribution; At any time The process of sampling is represented as: wherein is the attenuation coefficient associated with , represents the product of all forward diffusion processes. 6.The medical image segmentation method based on the region-border mutual learning diffusion model according to claim 1, characterized in that, The specific process of step 4 is: Step 4.1, generating a noise mask map and a pre-processed unlabeled medical image to be segmented As an input of the conditional encoder, the conditional encoder comprises four stages, the first stage comprises a fusion embedding layer and a Transformer encoder, the Transformer encoder of the first stage outputs a feature map ; Step 4.2, the feature map As the second stage of the conditional encoder, the second stage of the Transformer encoder outputs a feature map ; Step 4.3, feature map input condition encoder third stage, third stage transformer encoder output feature map ; Step 4.4, feature map input condition encoder fourth stage, fourth stage transformer encoder output feature map .

7. The medical image segmentation method based on a region-border mutual learning diffusion model according to claim 1, characterized in that, The specific process of step 5 is: Step 5.1, respectively, the feature map 、 、 and Input to the four intra-layer feature interaction modules of the region perception module, realize different direction and scale feature interaction, four intra-layer feature interaction modules output feature map 、 、 and , recorded as , expressed as: In the formula, represent the first feature map generated by the main network, represent the number of branches, represent the output of the first branch, represent element addition, represent asymmetric convolution, represent convolution, represent convolution, represent the splicing of three branches, is ReLU an activation function; Step 5.2, input the feature maps ( , ) and ( , ) into two inter-layer feature coordination modules of the region perception module respectively to strengthen the shared feature expression, to obtain output feature maps and , represented as: wherein, represents two ordinary convolutions, represents a hyperbolic tangent function, represents norm, represents element-wise multiplication, represents matrix transpose, represents feature maps after two 1x1 convolution processes.

8. The medical image segmentation method based on a region-border mutual learning diffusion model according to claim 1, characterized in that, The specific process of step 6 is: Step 6.

1. Extracting boundary features of the unlabeled medical image to be segmented by Laplacian operator, to obtain a Laplacian boundary feature map is expressed as:​ wherein, represents an input unlabeled medical image to be segmented, represents a convolution operator with a Gaussian filter, represents down-sampling, represents up-sampling, represents a Laplacian image of the k layer of the Laplacian pyramid, represents an unlabeled medical image to be segmented feature map after Gaussian convolution and down-sampling; Step 6.2, generating a noise mask map The segmentation encoder of the input boundary-aware denoising network, and then inputting the output feature map of the segmentation encoder and the feature map After splicing and up-sampling, a predicted feature map is obtained ; Step 6.3, set the predicted feature map output by the region-boundary mutual learning module in the boundary-aware denoising network to , and take the inverse, highlighting the insignificant region, to obtain the reverse attention map , and take the inverse, highlighting the insignificant region, to obtain the reverse attention map , and take the inverse, highlighting the insignificant region, to obtain the reverse attention map , and take the inverse, highlighting the insignificant region, to obtain the reverse attention map is expressed as:​​​ wherein denotes the Laplacian operator; Step 6.

4. Cross-learning the regional feature maps by the region-boundary mutual learning module with the Laplacian boundary feature maps , the next-stage reverse attention maps and the boundary attention maps respectively, denoted as: In the formula, denotes a sigmoid activation function; Step 6.

5. Feature maps are output Refinement is performed within the channels and spaces, denoted as: wherein represent a channel attention operation, represent a spatial attention operation; Step 6.6, upsampling the result to obtain a result , repeating the region-boundary mutual learning module and the upsampling operation to obtain a prediction result . 9.The medical image segmentation method based on the region-border mutual learning diffusion model according to claim 1, characterized in that, The specific process of the reverse diffusion denoising in step 7 is: In the formula, and These represent the reverse diffusion process. and Noise mask at time step, Indicates in Time and current noise mask image The mean value predicted by the network model under the given conditions. This represents the variance of the Gaussian distribution during the backdiffusion process. These are the parameters that control Gaussian noise during the back diffusion process. Is with The relevant attenuation coefficient, Indicates all during the reverse diffusion process The product of This is a segmentation mask image.

Images