Pancreatic solid pseudopapillary tumor lesion segmentation method integrating diffusion model and voting mechanism

Abstract

The present invention discloses a method for segmenting solid pseudopapillary tumor lesions of the pancreas by integrating a diffusion model and a voting mechanism. The method comprises the following steps: Step 1, randomly dividing the case sample data of different patients into a training set or a test set, where the case sample data includes case images at different times and corresponding segmentation images; Step 2, adjusting each case image to the same size; Step 3, training a diffusion model based on the training set to converge; Step 4, using the diffusion model to obtain a mask prediction image of any case image in the test set; Step 5, after inputting the mask prediction images of the same patient at different times into the voting mechanism module for voting, obtaining a prediction result image; Step 6, through image scaling of the prediction result image, generating images of the same size as the segmentation images at each time period. It solves the special situation where the segmentation result is empty when the diffusion model is used for segmenting solid pseudopapillary tumor lesions of the pancreas, and at the same time improves the segmentation effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of image segmentation, and specifically relates to a method for segmenting solid pseudopapillary tumor lesions of the pancreas by integrating a diffusion model and a voting mechanism. Background Art

[0002] Solid pseudopapillary tumor of the pancreas (SPTP) is a rare benign or low-grade malignant exocrine gland epithelial tumor of the pancreas. Its overall incidence is not high, accounting for about 0.13% - 2.7% of pancreatic tumors and about 1% - 2% of pancreatic exocrine tumors. However, although the early malignancy of solid pseudopapillary tumor of the pancreas is not very high, its clinical symptoms are not obvious, and it often leads to misdiagnosis in the initial diagnosis or even multiple diagnoses, thus delaying the best time for cancer treatment of patients, allowing cancer more time to develop and spread, and causing the disease to deteriorate further.

[0003] In the image analysis of solid pseudopapillary tumor screening of the pancreas, due to the rarity, diversity and insufficient understanding of the disease, radiologists have difficulty accurately detecting and analyzing masses, and it is difficult to determine the nature of masses in the upper abdomen. In the actual imaging screening of solid pseudopapillary tumor of the pancreas, computed tomography (CT) is a relatively affordable and effective imaging technology-assisted diagnostic tool. For CT images, there are also four stages, namely the arterial phase, the portal phase, the parenchymal phase and the plain scan.

[0004] As a machine learning theory, deep learning aims to build and simulate a neural network for analyzing and learning in the human brain. It imitates the mechanism of the human brain to interpret data such as images, sounds and texts. The essence of deep learning is to form more abstract high-level representation attribute features by combining low-level features to discover the distributed feature representation of data. Therefore, deep learning is also called representation learning. The main previous methods for image segmentation are two methods: convolutional neural network (abbreviation: CNN) and Transformer. Among them, the representative methods based on CNN include Unet and various improved versions of Unet, such as: Unet++, U2Net, etc., while Transformer mainly includes TransUnet, etc.

[0005] In recent years, in the field of generation, diffusion models (abbreviation: Diffusion) have replaced generative adversarial networks (abbreviation: GAN) as the most popular research direction. The essence of Diffusion is reverse modeling based on the diffusion process and noise signals, and the original data distribution is gradually restored by iteratively applying diffusion and denoising operations. Diffusion has been applied in image generation, image denoising, super-resolution reconstruction, image restoration, and image processing and enhancement. And recently, Diffusion has also been applied in the field of image segmentation, such as SegDiff, etc. However, there are some deficiencies in existing research:

[0006] (1) The performance achieved by methods based on CNN and Transformer in the field of medical image segmentation is relatively limited;

[0007] (2) There is no relevant report on applying diffusion models to the segmentation of solid pseudopapillary tumor lesions in the pancreas, and when using diffusion models for lesion segmentation, the segmentation result may be empty, while this does not happen with CNN and Transformer. Summary of the Invention

[0008] The purpose of the present invention is to provide a method for segmenting solid pseudopapillary tumor lesions in the pancreas that combines a diffusion model and a voting mechanism, and a voting mechanism module is added to the diffusion model to solve the special situation where the segmentation result is empty during image segmentation by the diffusion model.

[0009] To achieve the above purpose, the present invention provides the following technical solutions:

[0010] A method for segmenting solid pseudopapillary tumor lesions in the pancreas that combines a diffusion model and a voting mechanism, the method comprising the following steps:

[0011] Step 1, randomly divide the case sample data of different patients into a training set or a test set, and the case sample data includes case images and corresponding segmentation images at different times;

[0012] Step 2, adjust each case image to the same size;

[0013] Step 3, train the diffusion model based on the training set until convergence;

[0014] Step 4, use the diffusion model to obtain the mask prediction image of any case image in the test set;

[0015] Step 5, after passing the mask prediction images of the same patient at different times into the voting mechanism module for voting, obtain the prediction result image;

[0016] Step 6: Generate images of the same size as the segmentation images of each period by image scaling of the predicted result image.

[0017] Preferably, the voting of the voting mechanism module includes the following steps:

[0018] After removing the abnormal mask prediction images, calculate the voting threshold based on the number of mask prediction images of the same patient;

[0019] Calculate the pixel sum of the same pixel point and compare it with the voting threshold. If it is greater than or equal to the voting threshold, the value of this pixel point in the predicted result image is 1, otherwise it is 0.

[0020] Preferably, the voting threshold is rounded up to half of the number of mask prediction images of the same patient.

[0021] Preferably, the diffusion model uses a noise reduction model based on the Unet architecture to fit the noise ε to obtain the noisy image y at the (t - 1)th step from the noisy image y at the tth step; t get the noisy image y at the (t - 1)th step t-1 ;

[0022] The noise reduction model includes a pre - placed Unet and a post - placed SE Block.

[0023] Preferably, the Unet includes two parallel and identical encoders and a decoder symmetric to the encoder; the encoder includes 4 convolutional modules arranged in sequence. The first 3 convolutional modules each include a convolutional layer and a linear attention mechanism layer, and the 4th convolutional module includes a convolutional layer and a self - attention mechanism layer; the two encoders respectively downsample the case image and the noisy image y at the tth step step by step and save each downsampled feature map; the outputs of the two encoders are concatenated in the channel dimension as the input of the first convolutional module of the decoder and are upsampled. The newly upsampled feature map in the decoder is concatenated with the downsampled feature map at the corresponding position in the encoder in the channel dimension and is sequentially passed into the latter 3 convolutional modules. t The outputs of the two encoders are concatenated in the channel dimension as the input of the first convolutional module of the decoder and are upsampled. The newly upsampled feature map in the decoder is concatenated with the downsampled feature map at the corresponding position in the encoder in the channel dimension and is sequentially passed into the latter 3 convolutional modules.

[0024] Preferably, in the encoder corresponding to the case image, feature correlation layers are respectively arranged after the first 2 convolutional modules. The feature correlation layer is associated with the downsampled feature map output by the same convolutional module in the other encoder, and is used to multiply the output of the convolutional module after suppressing the noise of the associated downsampled feature map through the FFP module.

[0025] Preferably, step 4 includes:

[0026] Repeatedly input each case image in the test set into the diffusion model to obtain multiple output images, take the pixel average of the pixel points at the same position, and update the pixel value of the pixel point based on the pixel threshold to obtain the mask prediction image.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] 1. Apply the diffusion model to the field of segmentation of pancreatic solid pseudopapillary tumor lesion images, improve the diffusion model, add the SE Block (Squeeze-Excitation Block) and the FFP module (Feature Frequency Parser module) to better extract features from pancreatic solid pseudopapillary tumor data, and adopt a linear attention mechanism to reduce the model complexity.

[0029] 2. Add a voting mechanism module to the diffusion model to perform pixel voting on case images at different times and obtain the final segmentation result, solve the special case where the segmentation result is empty when the diffusion model is used for pancreatic solid pseudopapillary tumor lesion segmentation, and improve the segmentation effect at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flowchart of the method of the present invention;

[0031] Figure 2 is a schematic diagram of the training process principle of the diffusion model in the present invention;

[0032] Figure 3 is a schematic diagram of the test process principle of the method of the present invention;

[0033] Figure 4 is a schematic diagram of the FFP module principle in the diffusion model of the present invention;

[0034] Figure 5 is a schematic diagram of the SE Block module principle in the diffusion model of the present invention;

[0035] Figure 6 is a comparison diagram of the method of the present invention with other models. DETAILED DESCRIPTION OF THE INVENTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0037] Refer to Figure 1 , a method for segmenting pancreatic solid pseudopapillary tumor lesions by integrating a diffusion model and a voting mechanism, and this method includes 6 steps.

[0038] Step 1, randomly divide the case sample data of different patients into a training set or a test set. The case sample data includes case images at different times and corresponding segmentation images.

[0039] Generally, each patient has four different-phase case images during the CT process, namely plain scan, arterial phase, portal vein phase, and parenchymal phase. Although the different case images are for different phases of each patient, they are for the same part of the patient. Since the display effects of the case images in different phases are different, it will have a great impact on the doctor's judgment.

[0040] In the present invention, a case sample data includes at least three-phase case images and their corresponding segmentation images, otherwise it does not constitute case sample data. The case images are all jpg pictures converted from CT images; experts manually perform image segmentation on the case images to mark the lesion areas, thereby obtaining a segmentation result mask image, that is, the segmentation image, and the segmentation image is a binary image.

[0041] Step 2: Adjust each case image to the same size.

[0042] In the present invention, since the sizes of the collected case images are not the same, and the case images of the same patient in different phases are also different, while the diffusion model has a fixed requirement for the size of the input image, it is necessary to adjust each case image to the same size.

[0043] Step 3: Train the diffusion model based on the training set to converge.

[0044] The diffusion model includes forward propagation and backward propagation. Forward propagation is the forward noise addition process, and backward propagation is the backward denoising process. Taking a total of T steps as an example: In the forward propagation process, the whole process is a Markov chain process, and the image is noise-added until a completely random normal distribution image is obtained; in the backward propagation process, denoising starts from step T, and the generated picture can be obtained by iterating to step 0. Give the sample a random time value t between 0 and T. This time t represents the t-th step of noise addition and denoising in the diffusion model. In the following specific downsampling and upsampling processes, using time encoding can let the diffusion model know which time step t this is. In the specific t-th step of noise addition and denoising, a denoising model with a Unet architecture is used to fit the noise distribution at time t.

[0045] The forward propagation of the diffusion model is specifically formulated as follows:

[0046]

[0047]

[0048] Where T is the total number of steps of the diffusion model, x1...x tis the intermediate latent variable with added noise, x0 is the original input of the case image, q is the forward propagation process, and the noise addition process in the forward propagation process is a Markov chain process. The specific Markov chain noise addition process is shown in Equation 2, that is, adding noise with a random normal distribution after multiplying by the coefficient and then adding the coefficient β t-1 Multiply by the coefficient and then add the coefficient β t of the random normal distribution, where β t is a continuous parameter with a length equal to t. At the same time, according to β t we can calculate α t = 1 - β t , Since the forward propagation process is a Markov chain process, we can directly skip the intermediate iteration process and directly obtain the distribution at time t based on the distribution at time 0. The specific formula is as follows:

[0049]

[0050] In the actual forward propagation process, the diffusion model randomly selects a time t from 0 to T, and then adds noise ε to obtain x t .

[0051] In the backward propagation process, this process is still a Markov chain process. The specific formula is as follows:

[0052]

[0053] where the formula starts with p θ (x t ) = N(x t ; 0, I n×n ), that is, x t is a random normal distribution. The reverse process iterates the normal distribution p θ (x T ) to the final segmentation result p θ (x0). Specifically, in terms of the formula:

[0054]

[0055] The parameters are:

[0056]

[0057]

[0058] In this way, we obtain the mean and variance at the t - 1 moment in the forward propagation process, and then we can obtain the distribution at the t - 1 moment. Then, inputting the distribution at the t - 1 moment into the noise reduction model of the Unet architecture, we can obtain the noise distribution at t - 2, and through calculation, we can obtain the distribution at t - 2. Repeating this process until time 0, we can obtain the final result.

[0059] During the reverse denoising process, a denoising model with a Unet architecture is used to fit ε to train the network and make the model converge.

[0060] The denoising model with the Unet architecture includes a front Unet and a rear SE Block.

[0061] U-net is a typical encoding-decoding structure. In the encoder part, pooling layers are used for successive downsampling, and in the decoder part, transposed convolutions are used for successive upsampling. The spatial information and edge information in the original input image will be gradually restored. Thus, the low-resolution feature map will finally be mapped to a pixel-level segmentation result map. To further compensate for the information lost during downsampling in the encoding stage, between the encoder and decoder of the network, the U-Net algorithm uses skip connections to fuse the feature maps at corresponding positions in the two processes, enabling the decoder to obtain more high-resolution information during upsampling, thereby more perfectly restoring the detailed information in the original image and improving the segmentation accuracy.

[0062] The method of the present invention is improved on the traditional Unet for fitting noise ε. Its encoder includes 4 convolutional modules arranged in sequence. The first 3 convolutional modules each include a convolutional layer and a linear attention mechanism layer, and the 4th convolutional module includes a convolutional layer and a self-attention mechanism layer; the two encoders respectively perform successive downsampling on the case image and the noisy image y at the t-th step t and save each downsampled feature map; the decoder includes 4 convolutional modules arranged in sequence. The first convolutional module includes a convolutional layer and a self-attention mechanism, and the last 3 convolutional modules each include a convolutional layer and a linear attention mechanism layer; the outputs of the two encoders are concatenated in the channel dimension and used as the input to the first convolutional module of the decoder, and then upsampling is performed. The newly upsampled feature map is concatenated with the downsampled feature map at the corresponding position in the encoder in the channel dimension and input into the second convolutional module. The same operation is performed on the third and fourth convolutional modules.

[0063] In the encoder corresponding to the case image, feature correlation layers are respectively arranged after the first 2 convolutional modules. The feature correlation layer is associated with the downsampled feature map output by the same convolutional module in the other encoder, and is used to multiply the output of the convolutional module after suppressing the noise of the associated downsampled feature map through the FFP module. The FFP module can constrain the noise part in the noisy image y t at time t, making the non-noise part more distinguishable. This is manifested in the figure as the non-noise part and the noise part being more easily distinguishable. Combining Figure 4 to illustrate the specific implementation process of the feature correlation layer: y tThe tensor is mapped to the Fourier space, then multiplied by a learnable tensor to obtain the processed data. Finally, it is mapped from the Fourier space back to the normal space to obtain a tensor of yt with noise suppressed. Then, it is subjected to the LayerNorm operation, and at the same time, the feature tensor of x at the corresponding position is also subjected to the LayerNorm operation. Finally, the two results are multiplied to obtain the final result, which is used as the feature tensor of x for subsequent downsampling operations.

[0064] Using the self-attention mechanism in high-dimensional features can fully learn global features to help capture richer context information and improve the accuracy of image segmentation. A linear attention mechanism layer (Linear Attention) is set in low-dimensional features to fuse the features after passing through the convolutional layer. The linear self-attention mechanism can greatly reduce the computational complexity while effectively capturing the context information and fine-grained features in the image.

[0065] This SE Block performs channel attention mechanism on the features finally extracted by Unet to learn the importance of different channels, obtaining the noise ε at time t; referring to Figure 5 As shown, the implementation principle of this SE Block is as follows:

[0066] First, the input feature map is compressed through global average pooling. The global average pooling operation calculates the average value of each feature channel in the spatial dimension to obtain a feature vector. Next, a fully connected layer is used to learn the relationship between feature channels. This fully connected layer usually consists of two linear transformations, namely a dimensionality reduction operation and a dimensionality increase operation. The dimensionality reduction operation compresses the feature vector into a smaller dimension, and then introduces non-linearity through an activation function; the dimensionality increase operation maps the dimensionality-reduced feature vector back to the dimension of the original feature channel. Finally, the output result of the excitation operation is used as the attention weight, and each feature channel is multiplied by the corresponding attention weight to scale the feature channel. In this way, the importance of each feature channel will be adaptively adjusted according to the learned attention weight.

[0067] In the present invention, by setting the SE Block, the importance of each channel can be dynamically learned and scaled according to these weights, which helps the network better capture the key features of the network.

[0068] Step 4: Use the diffusion model to obtain the mask prediction image of any case image in the test set. Specifically, each case image in the test set is repeatedly input into the diffusion model for multiple samplings. The average pixel value is taken for the pixel points at the same position, and a pixel threshold of 0.5 is set. The value of the pixel points greater than or equal to 0.5 is set to 1, and the value of the pixel points less than 0.5 is set to 0. In this way, the final mask prediction image is obtained.

[0069] Step 5: After passing the mask prediction images of the same patient at different times into the voting mechanism module for voting, the predicted result image is obtained.

[0070] The voting of the voting mechanism module includes the following steps:

[0071] Step 5.1: After removing the abnormal mask prediction images, calculate the voting threshold based on the number of mask prediction images of the same patient. The voting threshold is the ceiling of half of the number of mask prediction images of the same patient.

[0072] Step 5.2: Calculate the pixel sum of the same pixel point and compare it with the voting threshold. If it is greater than or equal to the voting threshold, the value of the pixel point in the predicted result image is 1, otherwise it is 0.

[0073] In the present invention, the mask prediction image obtained through Step 4 may be abnormal. The abnormal mask prediction image is the case where all the pixel values of the image are 0. At this time, the abnormal mask prediction image needs to be removed. In addition, there is also a situation where the collected case images of the same patient are missing in time periods, resulting in only 3 corresponding mask prediction images. For the two situations, the voting threshold is calculated based on the number of mask prediction images as the basic data, so as to solve the special situation where the segmentation result is empty when the diffusion model performs image segmentation. It should be noted that after passing the mask prediction images of the same patient at different times into the voting mechanism module, there is exactly 1 predicted result image.

[0074] Step 6: Generate the same-sized images of the segmentation images of each time period by image scaling of the predicted result image.

[0075] In the present invention, since the sizes of the original segmentation images are different, the predicted result image is restored to the same size as the segmentation image of the corresponding time period through image scaling to obtain the corresponding same-sized images, and then the performance of the evaluation model is evaluated based on the segmentation image and the same-sized image in the same time period. It should be noted here that there is 1 predicted result image. Through multiple image scalings of the predicted result image, multiple images are obtained. The number of scalings is determined by the number of case images of the same patient, and the size of the scaled image is determined by the sizes of the segmentation images of the same patient at each time period.

[0076] In the present invention, an image segmentation method MMSDiff that combines a diffusion model (Diffusion model) and a voting mechanism module (vote) solves the special case where the segmentation result is empty during image segmentation by the diffusion model, and improves the segmentation effect of solid pseudopapillary tumor lesions of the pancreas.

[0077] Example 1:

[0078] Step 1: Collect case sample data of 113 patients, including 336 case images and corresponding 336 segmentation images; divide the case sample data so that the quantity ratio of case sample data between the training set and the test set is 7:3. There are 253 case images in the training set and 83 case images in the test set.

[0079] Step 2: The lengths and widths of the 336 case pictures collected in this dataset are not the same, and the length and width are approximately between 900 and 1100. Therefore, data preprocessing needs to be performed during data training to process them into tensors with fixed length and width. Adjust the case images in the training set and the test set to a length and width of 128 each through bilinear interpolation, denoted as [128, 128].

[0080] Step 3: Input the training set into the diffusion model to train the model to convergence.

[0081] Because the diffusion model has a high degree of randomness, a large number of iteration times are required. During the training process, since each sample randomly selects a time t within T times for training, and does not train the other time states between 0 and t, but directly calculates the noise ε between y0 and y t and uses the model to fit this noise ε, a large number of iteration steps are required to make the model converge and obtain good results. The optimizer used during training is the Stochastic Gradient Descent optimizer, the learning rate of the optimizer is 0.001, the momentum is 0.9, and the weight decay is 0.0005. The total number of training steps can be adjusted according to different situations.

[0082] In this embodiment, the tensor y after adding noise to the mask at time t tThe shape is: [b, c, h, w] = [1, 1, 128, 128], where b represents the size of batch_size, c represents the number of channel channels, h represents the height of the image, and w represents the width of the image. A tensor of [1, 64, 128, 128] is obtained through the first convolutional module in the corresponding encoder. This tensor undergoes three downsampling operations through the remaining three convolutional modules, and at the same time, this tensor is used as the input to the FFP module to obtain the output of the FFP module. The shape of the tensor of the case picture is: [b, c, h, w] = [1, 3, 128, 128]. A tensor of [1, 64, 128, 128] can be obtained through the first convolutional module in the corresponding encoder. This tensor is multiplied pointwise with the output of the corresponding FFP module to obtain the pointwise multiplication result of [1, 64, 128, 128]; this pointwise multiplication result undergoes downsampling encoding through the remaining three convolutional modules in the corresponding encoder to obtain tensors of [1, 128, 64, 64], [1, 256, 32, 32], and [1, 512, 16, 16]. It should be noted here that in the encoder corresponding to the case image, the downsampled feature map of [1, 128, 64, 64] obtained is the output result of the feature correlation layer set after the second convolutional module. In the present invention, temporal encoding is to be performed during the downsampling encoding process.

[0083] Concatenate the outputs of the two encoders in the channel dimension, that is, concatenate two tensors of [1, 512, 16, 16] to form a tensor of [1, 1024, 16, 16]. Input the tensor of [1, 1024, 16, 16] into the first convolutional module of the decoder to obtain a feature tensor of [1, 512, 16, 16]. Perform an upsampling operation to obtain a feature tensor of [1, 256, 32, 32]. Then, after concatenating the [1, 256, 32, 32] in the downsampled feature map, input it into the second convolutional module of the decoder to obtain a feature tensor of [1, 256, 32, 32]. Perform the same upsampling operation to obtain a feature tensor of [1, 128, 64, 64]. After concatenating the [1, 128, 64, 64] in the downsampled feature map, input it into the third convolutional module of the decoder to obtain a feature tensor of [1, 64, 128, 128]. Use it as the input of the fourth convolutional module to obtain a feature tensor of [|1, 64, 128, 128|] and pass it into the SE Block (Squeeze-and-Excitation Block) to perform channel self-attention operation on the tensor. The specific self-attention operation is as follows: perform global pooling on [1, 64, 128, 128] in the channel dimension to obtain a tensor of [1, 64, 1, 1]. Then, perform a fully connected layer learning on [1, 64, 1, 1]. After obtaining a tensor with channel attention, multiply [1, 64, 128, 128] and [1, 64, 1, 1] to obtain a feature map with channel attention. Finally, pass the obtained result through a residual convolutional block to obtain a tensor of [1, 1, 128, 128]. At this time, the obtained tensor can be used as the noise ε at time t simulated by the denoising model.

[0084] The SE Block is an attention mechanism that helps improve the expression ability of the convolutional neural network in the feature channel dimension by adaptively learning the importance weights between channels, thereby improving the performance and generalization ability of the model.

[0085] Step 4: Input the case images in the test set into the trained diffusion model. Each time, input a single case image, and obtain an output through the diffusion model. Repeat inputting the same case image to obtain 5 outputs of the same case image on the diffusion model. Perform an averaging operation on these 5 output results in the channel dimension. Finally, obtain the average value of 5 samplings, and then compare it with the threshold of 0.5. Those greater than the threshold are set to 1, and those less than the threshold are set to 0.

[0086] Step 5: After the mask prediction images of the same patient at different times are passed into the voting mechanism module for voting, obtain the predicted result image.

[0087] Step 6: Scale the predicted result image through image scaling to generate images of the same size as the segmentation images at each time period.

[0088] In this embodiment, in order to verify the effectiveness of the method MMSDiff of the present invention in pancreatic solid pseudopapillary tumor image segmentation, the performance of the diffusion model is evaluated based on the segmented images of each period of cases and their images of the same size. Specifically, the Dice, mlou, and Accuracy parameters are calculated through the segmented images in the test set and the corresponding images of the same size, so as to evaluate the performance of the model.

[0089] In the present invention, the method of this embodiment is compared with other typical deep learning models, such as Unet(ResNet), Unet(VGG), Unet++, U2Net, TransUnet, SegDiff, MedSegDiff and other models in terms of segmentation performance. For conventional methods based on CNN and Transformer, a voting mechanism module is combined after obtaining the segmentation results, and those based on the Diffusion model use the original method, which can better illustrate the improvement of the voting mechanism module for the Diffusion model. Among them, Unet(ResNet) uses Resnet18 and Resnet50 as the models of the feature root extractor respectively, Unet(VGG) is a model with VGG as the feature extractor, Unet++ is an improved multi-level feature extractor image segmentation model of Unet, U2net is also an improved Unet nested feature extractor image segmentation model of Unet, TransUnet is an image segmentation network combining Transformer and CNN, and SegDiff and MedSegDiff are two models applying the diffusion model in image segmentation.

[0090] Figure 6 The segmentation performances of various methods in image segmentation are given. It can be seen from the figure that the method MMSDiff of the present invention has better performance than other segmentation networks, that is, the Dice, mlou, and Accuracy indexes reach 66.59%, 57.64%, and 99.01% respectively.

[0091] The above embodiments only represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

[0092] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art will appreciate that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for segmenting solid pseudopapillary tumor lesions of the pancreas that combines a diffusion model and a voting mechanism, characterized in that, The method includes the following steps: Step 1: Randomly divide the case sample data of different patients into a training set or a test set. The case sample data includes case images and corresponding segmentation images at different times. Step 2: Adjust each case image to the same size. Step 3: Train a diffusion model based on the training set until convergence. Step 4: Use the diffusion model to obtain the mask prediction image of any case image in the test set. Step 5: After inputting the mask prediction images of the same patient at different times into the voting mechanism module for voting, obtain the prediction result image. Step 6: Through image scaling of the prediction result image, generate images of the same size as the segmentation images at each time period respectively. The voting of the voting mechanism module includes the following steps: After removing abnormal mask prediction images, calculate the voting threshold based on the number of mask prediction images of the same patient. Calculate the pixel sum of the same pixel point and compare it with the voting threshold. If it is greater than or equal to the voting threshold, the value of this pixel point in the prediction result image is 1, otherwise it is 0. The diffusion model uses a noise reduction model based on the Unet architecture to fit the noise ε to obtain the noisy image y at the t-th step t to obtain the noisy image y at the (t-1)-th step t-1 ; The noise reduction model includes a pre - placed Unet and a post - placed SE Block. The Unet includes two parallel and identical encoders and a decoder symmetric to the encoder. The encoder includes 4 convolutional modules arranged in sequence. The first 3 convolutional modules each include a convolutional layer and a linear attention mechanism layer, and the 4th convolutional module includes a convolutional layer and a self - attention mechanism layer. The two encoders respectively perform downsampling on the case image and the noisy image yt at the t - th step level by level and save each downsampled feature map. The outputs of the two encoders are concatenated in the channel dimension as the input of the first convolutional module of the decoder, and then upsampled. The newly upsampled feature map in the decoder is concatenated with the downsampled feature map at the corresponding position in the encoder in the channel dimension, and then sequentially passed into the latter 3 convolutional modules. In the encoder corresponding to the case image, feature correlation layers are respectively set after the first 2 convolutional modules. The feature correlation layer is associated with the downsampled feature map output by the same convolutional module in the other encoder, and is used to multiply the output of the convolutional module after suppressing noise of the associated downsampled feature map through the FFP module.

2. The method for segmenting solid pseudopapillary tumor lesions of the pancreas by integrating a diffusion model and a voting mechanism according to claim 1, characterized in that, The voting threshold is the ceiling of half of the number of mask prediction images of the same patient.

3. A method for segmenting solid pseudopapillary tumor lesions of the pancreas that integrates a diffusion model and a voting mechanism, characterized in that, Step 4 includes: Repeatedly input each case image in the test set into the diffusion model to obtain multiple output images. Take the pixel average of the pixel points at the same position and update the pixel value of the pixel point based on the pixel threshold to obtain the mask prediction image.

Images