Feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodule

Abstract

The invention relates to the technical field of image processing, in particular to an ultrasonic thyroid nodule feature pyramid edge weighted diffusion segmentation method, which comprises the following steps: acquiring an ultrasonic thyroid nodule image; a PEW-SegDiff model is constructed; gaussian noise is added through time in the forward diffusion process of the real mask image, and a label with noise is obtained; in the back diffusion process, the input image and the real mask image after forward diffusion are added to obtain a noise label feature image; training a PEW-SegDiff model by using the noise tag feature graph and the input graph to obtain a predicted initial image; and calculating the loss between the predicted image and the back diffusion image of the real mask. The thyroid nodule edge blurred image extraction method solves the problems that an original SegDiff model is limited in boundary learning capability and the effect is poor when thyroid nodule edge blurred images are extracted.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules. Background Art

[0002] Ultrasound images contain various inherent artifacts and noises, such as speckle noise, blurred boundaries, uneven intensity, etc., which will affect the accuracy of model segmentation; among them, boundary information is crucial for lesion segmentation because it helps the network distinguish between lesions and normal tissues. However, the original diffusion segmentation network has limited ability to learn boundaries. Due to the low contrast of ultrasound images and the blurred edges of thyroid nodules, boundary information is crucial.

[0003] In the field of segmentation, Amit et al. pioneered the development of an end-to-end diffusion-based medical image segmentation network that does not rely on pre-training; Wu et al. introduced Fourier transform to eliminate the negative impact of high-frequency noise in the model learning process to improve the accuracy; Aimon et al. proposed a method based on a single diffusion model to achieve better performance than existing fuzzy segmentation networks or fuzzy medical image segmentation; Xing et al. proposed the DiffUNet model, which improves the robustness of the model by introducing a step-size uncertainty-based encoding module and a fusion (SUF) module; Chen et al. proposed using Bernoulli noise as a diffusion kernel to enhance the ability of the diffusion model for binary segmentation tasks; Chowdary, GJ et al. proposed a Transformer model based on U-Net, which uses windows of multiple sizes to extract contextual information of various scales, thereby improving the pixel-level segmentation performance of the diffusion model.

[0004] Although researchers have verified the effectiveness of diffusion probability models for different medical image analysis tasks, few tasks use ultrasound image datasets, so the problems of ultrasound images in diffusion models have not yet been discovered. Summary of the invention

[0005] To address the shortcomings of existing methods, the present invention proposes a multi-branch diffusion segmentation network (PEW-SegDiff), which introduces pyramid edge weighting (PEW) to focus on edge information; and proposes a multi-layer weighted model (MLW) to connect multi-scale features to capture and weight boundary information, thereby improving the accuracy of model segmentation; and uses the total loss function to further improve the classification accuracy.

[0006] The technical solution adopted by the present invention is: a characteristic pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules comprises the following steps:

[0007] Step 1: Acquire ultrasonic thyroid nodule image I;

[0008] Step 2: During the forward diffusion process, Gaussian noise is added to the real mask image X0 to obtain a noisy label; during the reverse diffusion process, the input image I and the real mask image X0 after forward diffusion are added to obtain the noise label feature image X t ; Using the noise label feature map X t The PEW-SegDiff model is trained with the input image I to obtain the predicted initial image; the loss between the predicted image and the reverse diffusion image of the true mask is calculated.

[0009] As a preferred embodiment of the present invention, the PEW-SegDiff model includes: a pyramid edge weighting module and a denoising network; wherein the pyramid edge weighting module is a dual-channel, the first channel includes: inputting the feature map into the feature pyramid module after resizing, and then performing group normalization to obtain a first channel feature map; the second channel includes: obtaining a second channel feature after performing double convolution and group normalization on the feature map; the first channel and the second channel features are combined and input into a multi-layer weighting module, and then element-wise multiplication is performed with the second channel feature map through an activation function.

[0010] As a preferred implementation of the present invention, double convolution is composed of 3×3 convolution and 1×1 convolution, and a time vector is added between the two convolution blocks.

[0011] As a preferred embodiment of the present invention, the multi-layer weighted module includes: the feature map is respectively input into three parallel convolutions with convolution kernels of 1x1, 3x3, and 3x3, and then the features are output, and the features are combined and resized to obtain the weight coefficient.

[0012] As a preferred embodiment of the present invention, the ASPP layer in the feature pyramid module is improved to obtain the AT-ASPP layer, which includes: after 1x1 convolution and addition operation with the time vector, 1x1 convolution, 3x3rate=18 convolution, 3x3rate=12 convolution, 3x3rate=12 convolution are respectively input and then the convolution is combined, and then input into the S-Att layer.

[0013] As a preferred embodiment of the present invention, the formula for the loss between the predicted image and the reverse diffusion image of the true mask is:

[0014] L total =L dice +αL U +βL edge (7)

[0015] Among them, L dice To calculate the Dice loss of the fused image, α and β are different weights, and L U is the MSE loss of the main path, L edgeis the MSE loss of the edge path.

[0016] As a preferred embodiment of the present invention, L U The formula is:

[0017]

[0018] Among them, M pred-U is the prediction result of the main path, represents the MSE loss function, M GT is the time X obtained by back diffusion t-1 The segmentation mask of the ground-truth mask.

[0019] As a preferred embodiment of the present invention, L edge The formula is:

[0020]

[0021] Among them, M pred-edge It is the prediction result of edge branch prediction.

[0022] As a preferred embodiment of the present invention, a feature pyramid edge weighted diffusion segmentation system for ultrasonic thyroid nodules includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement a feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules.

[0023] As a preferred embodiment of the present invention, a computer readable medium stores computer program code, and when the computer program code is executed by a processor, the method for feature pyramid edge weighted diffusion segmentation of ultrasonic thyroid nodules is implemented.

[0024] Beneficial effects of the present invention:

[0025] 1. For the TN3K public dataset, PEW-SegDiff achieved a Dice coefficient (DSC) of 78.14% and an IoU of 74.48%; for the private dataset, DSC reached 78.59% and IoU reached 74.09%;

[0026] 2. Considering the importance of boundary information in the training process, a pyramid edge weighting module is proposed. The pyramid module is used to extract boundary features, which are weighted to the main feature map after multi-scale weight calculation, solving the problem of uneven edges of the predicted mask image during diffusion segmentation;

[0027] 3. In the edge learning branch, the image is first resized to the same size as the current layer, and low-level feature information such as edge structure is captured through the feature pyramid;

[0028] 4. After using multi-layer weighted calculation to calculate the weight of the learned edge features, it is weighted into the feature map of the denoising network to increase the model's attention to the edges;

[0029] 5. In the middle part of the model, an improved dilated convolution module is introduced according to the requirements of the diffusion model. By adding a time step vector, the training process of the dilated convolution and the diffusion segmentation network is made more consistent; the introduction of the hole convolution expands the receptive field of the edge learning branch, so that the edge features of this branch can be better combined with the spatial information features in the main branch. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a block diagram of the feature pyramid edge weighted diffusion segmentation model of the present invention;

[0031] Figure 2 This is the effect diagram of the diffusion segmentation model of different thyroid ultrasound images;

[0032] Figure 3 The original SegDiff diffusion segmentation architecture diagram;

[0033] Figure 4 This is the PEW-SegDiff segmentation network architecture diagram;

[0034] Figure 5 It is a detailed structural diagram of the pyramid edge weighting module;

[0035] Figure 6 This is a schematic diagram of adding time vectors in Atrous Spatial PyramidPooling;

[0036] Figure 7 is a comparison diagram of the segmentation results of the existing method and the method of the present invention;

[0037] Figure 8 It is a comparison diagram of the loss function changes during the training process of the existing method and the method of the present invention;

[0038] Fig. 9 is the predicted mask layout with noise. DETAILED DESCRIPTION

[0039] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0040] The feature pyramid edge-weighted diffusion segmentation method for ultrasound thyroid nodules includes the following steps:

[0041] like Figure 2As shown in the figure, the sample data comes from the TN3k dataset. Through experiments, it is found that the original diffusion model (segDiff) has the problem of blurred edges when performing ultrasound image segmentation. Figure 2 The third column shows that segDiff has inaccurate boundaries and is not smooth regardless of size. The fourth column shows that the diffusion segmentation model of PEW-SegDiff of the present invention can effectively solve the problem of inaccurate boundaries.

[0042] Figure 3 For the existing SegDiff segmentation framework, SegDiff focuses on denoising the noisy segmentation mask of the input image, so it adopts an attention segmentation module that focuses on spatial information, while ignoring the boundary information that is also important for medical image segmentation; Figure 1 The present invention proposes to add boundary information in the learning process to improve the segmentation performance; the edge weighted learning module weights the edge information to the important nodes of the SegDiff denoising network to increase the overall network's attention to edge information. In order to better integrate multi-scale features, combined with the needs of the diffusion model, a dilated convolution pooling module with a time vector (AT-ASPP) is added.

[0043] The denoising network adopts a denoising segmentation model based on attention to spatial information, which mainly learns spatial information and the dependency between pixels. Therefore, the segmented image has many irregular edges, and the ultrasound image also has the disadvantage of unclear edges. In order to solve this problem, a novel denoising network is proposed. Figure 4 Schematic diagram of pyramid edge weighted segmentation network. Pyramid edge weighting (PEW) is used to make up for the deficiency of low-level feature learning in denoising network. The final output is the weighted fusion of the output results of two branches.

[0044] The main body of SegDiff is DDPM. The core idea is to use the time vector to learn the distribution of data. The diffusion process gradually changes the noise vector to approximate the real data distribution. The diffusion process mainly includes two processes: forward diffusion and reverse diffusion. In the forward diffusion, DDPM is transformed from the real real mask map X0∈R H×W×1 At the beginning, Gaussian noise is gradually added through t time steps to make it a noisy label. The forward diffusion formula is:

[0045]

[0046] Among them, N() is Gaussian noise; xt refers to the data at time t; I is the input graph; β1, ..., β t-1 , β t Represents the variance of each stage of the forward diffusion process; in the reverse diffusion process, given an input image I H×W×3As a guide, the DDPM is trained to predict the initial image and calculate the loss between the predicted image and the back-diffused image of the true mask; then, iterative steps are taken to gradually improve the prediction ability of the model as follows:

[0047] p θ (x t-1 |x t )=N(x t-1 ; μ(I t , t), ∑ θ (I t , t)) (2)

[0048]

[0049] Where θ is the parameter of back diffusion; I t ∈R H×W×(3+1) , that is, the noise label feature map X t .

[0050] From the Gaussian noise p θ At the beginning, gradually distribute the variable p θ (x t ) is converted to p θ (x0); according to the process of forward diffusion, the noise image is gradually restored, and finally a clear segmentation mask is obtained; the MSE value between the predicted mask and the true mask is calculated through the loss function as the basis for model optimization, and the MSE difference between the two is continuously reduced to obtain an accurate segmentation mask.

[0051] like Figure 5 This is the detailed structure diagram of PEW (Pyramid Edge Weighting). The edge information is extracted from the original image through the feature pyramid module, and then weighted to the feature map of the denoising network. The feature pyramid can effectively capture the edge position information of images at different scales. Therefore, the edge features at different scales extracted by the feature pyramid can effectively achieve the task of edge feature weighting. Through the multi-layer weighted module (three-branch convolution), each branch weights the downsampled image of the denoising network.

[0052] First, the original image I is processed and adjusted to the same scale as the currently passed in feature map. Then, the edge features are extracted through the feature pyramid module, and then group normalization is performed. The feature pyramid increases the number of layers as the number of downsampling increases, thereby learning a representation that is more consistent with the current scale and improving the model accuracy.

[0053] The image x passed to the denoising network iAfter concatenating with the upper-layer features saved in the path, two layers of convolution are used to achieve the effect of the residual function, alleviating the gradient problem caused by the use of two layers of convolution; the two layers of convolution are composed of 3×3 convolution and 1×1 convolution, and the time vector t is added between the two convolution blocks, which helps the model understand different diffusion steps, thereby improving the denoising effect. The process is expressed as follows:

[0054]

[0055] Among them, x i Input feature map for denoising network, x i-1 It is the feature map retained by the upper pyramid weighted module.

[0056] The feature map X t After normalization, it is concatenated with the normalized feature map of the feature pyramid module to obtain X c , use the multi-layer weighted module to calculate the weight and obtain the weight coefficient k;

[0057] The multi-layer weighted model consists of three parallel convolution branches with different convolution kernels and strides to capture feature information at different scales; the convolution kernels of the three parallel convolutions are 1x1, 3x3, and 3x3, and the corresponding strides are 1, 1, and 2 respectively; the feature information output by the three convolutions is combined and resized, and then converted into a weight coefficient k through a nonlinear function activation (Sigmond); finally, it is combined with X t Multiply to achieve weighted output X w , the formula is:

[0058]

[0059] Among them, k is the feature map with the number of channels after being calculated and resized by the multi-layer weighted module (MLW), and X t is the original feature map, is a multiplication operation.

[0060] The multiplication operation emphasizes important features and merges non-critical features. As an attention mechanism, the noise map predicted by the model at time X t-1 The edges are highlighted, while the rest are also darkened, but as the back diffusion proceeds, they will be filtered out by the model.

[0061] Add the time vector t to ASPP to get AT-ASPP. The denoising network uses the attention mechanism to capture spatial information. The branch pyramid edge module uses a simple pyramid feature convolution to mainly learn low-level features such as edges. The features learned by the two paths are significantly different. In order to alleviate the large gap in segmentation results caused by feature differences and affect model iteration, the AT-ASPP module is added to the branch pyramid edge module.

[0062] The ASPP module uses multiple dilated convolutions with different sampling rates to enhance the receptive field and improve the branch's understanding of feature information. Global pooling improves the overall feature representation and can better fuse edge features and spatial information. However, the time vector is an important factor affecting the prediction results of the diffusion model. Therefore, the AT-ASPP module adds a time vector to the ASPP to make the fused features more in line with the needs of the diffusion model. The spatial attention mechanism (S-ATT) is used instead of the original 1×1 convolution to improve the branch's perception of spatial information. Figure 6 The AT-ASPP process is expressed as follows:

[0063]

[0064] in, is the sum of feature concatenation at different sampling rates after embedding the time vector, represents element-wise multiplication to broadcast the attention weights along the spatial dimension. After obtaining the attention weights, they are multiplied with the original feature map and then reshaped.

[0065] The PEW-SegDiff network adopts the total loss function L total , including the main path (L U ) and the MSE loss of the edge path (L edge ) of the MSE loss; the weights of the two paths during training are different (α, β); in order to speed up training, the Dice loss of the fused image is calculated (L dice ) and add it to the total loss. When the total loss value does not change much during training, the Dice loss will be removed; and dynamic weights are added to adjust the weight of the MSE function according to the progress of training to adjust the training effect. The total loss function L total The formula is:

[0066] L total =L dice +αL U +βL edge (7)

[0067]

[0068] Among them, L total Define the total loss function, L dice To calculate the Dice loss of the fused image, L U MSE loss of the main path, L edge is the MSE loss of the edge path, α, β are different weights, represents the MSE loss function, M pred-U is the prediction result of the main path, M pred-edge is the prediction result of the edge branch prediction, M GTis the time X obtained by back diffusion t-1 The segmentation mask of the ground-truth mask.

[0069]

[0070] in, represents the MSE loss function, M GT is the time X obtained by back diffusion t-1 The segmentation mask of the ground-truth mask.

[0071] Experimental results:

[0072] The model of the present invention is implemented using the PyTorch library (version 1.13.0) and trained on a single NVIDIA 3080ti (12GB) image processing unit. The training process uses a batch size of 2 and an Adam optimizer with a learning rate of 0.0001; in the experiment, the diffusion step size T is set to 1000 and a linear noise scheduler is used. It takes about 5 minutes to train 500 steps. During the sampling process, the time for a single image to pass the prediction and obtain the predicted mask through back diffusion is about 40 seconds.

[0073] A public thyroid ultrasound dataset TN3K and two self-built thyroid datasets were used; the TN3K dataset contains 3493 grayscale ultrasound images of 2421 patients. The dataset is divided into a training set and a test set with an allocation ratio of 8:2; 600 images are randomly selected from the training set as the data required for training to simulate small sample data; the test dataset contains 614 images; during training, the data is resized to 256×256, and random noise is generated and spliced ​​with the data for training.

[0074] The self-built thyroid dataset consists of ultrasound images collected by Changzhou Second People's Hospital Affiliated to Nanjing Medical University. They come from three commercial scanners from Philips Healthcare in Best in the Netherlands, Siemens Healthcare in Erlangen, Germany, and GE Healthcare in Chicago, USA. One set of images was collected in 2020 and the other set of images was collected in 2023. The first set of images contains 328 ultrasound images, and the second set of images comes from 153 patients, totaling 306 images; during the training process, the training set and test set division is also 8:2; the mask is completed by a doctor with rich clinical experience in ultrasound. In order to remove patient privacy and other information irrelevant to the condition, the image and mask are cropped to 512×512. During model training, it will be adjusted to 256×256.

[0075] The PEW-SegDiff model of the present invention is qualitatively compared with advanced segmentation models, including FCN, Deeplabv3+ResNet50, U-Net, AttentionUNet, TransUNet, Swin-UNet and SegDiff; all models are trained from scratch without applying any pre-trained weights, and 600 images are randomly selected as the training dataset on the public dataset TN3K to simulate the situation of small samples.

[0076] from Figure 7 In the randomly selected test images, observe the segmentation results produced by other models. Figure 7 (ah) are FCN, Deeplabv3-Resnet50, U-Net, AttnUNet, ransUnet, Swin-Unet, SegDiff and the model of the present invention. When the training data is limited, the segmentation mask of the model of the present invention is closest to the ground truth, which proves the effectiveness of the model of the present invention. Whether it is a large object (rows: 3, 4, 9) or a small object (rows: 2, 5, 6); Figure 7 In the present invention, all the comparison models have some problems in the edge part, so it is necessary to strengthen the boundary information. Compared with the diffusion model SegDiff, the model of the present invention introduces a boundary weighting module to solve the problem of insufficient segmentation boundary caused by diffusion. For the diagnosis and treatment of thyroid nodules, boundary information is crucial. Therefore, the idea of ​​introducing boundary weighting also has certain clinical significance. It can be seen that other models have some insufficient segmentation, which is mainly caused by insufficient training data. The present invention is based on the diffusion segmentation model and mainly learns to predict the image of the next time period. Therefore, even if the amount of data is small, good results can still be obtained.

[0077] In terms of ultrasound image segmentation, the performance of the present invention is compared with that of existing methods;

[0078] Table 1 Performance comparison between the model of the present invention and the existing model

[0079]

[0080] Table 1 shows the performance analysis of the PEW-SegDiff model of the present invention on all three thyroid ultrasound image segmentation datasets. The quantitative results show that compared with other models, the model of the present invention achieves better segmentation performance in the evaluation indicators when the data volume is small, with an accuracy of 91.94%, an IoU of 74.48%, and a DSC of 78.14%. Due to the small training dataset, the Transformer model has no advantage, and the evaluation indicators are even worse than those of U-Net and its derivative models. Compared with the original SegDiff, the model of the present invention achieves 2.01%, 1.74%, and 1.25% improvements in accuracy, IoU, and DSC, respectively.

[0081] On private dataset 1, since they all come from the same device, there is no significant difference in the quality of the collected images; the number of images used for training is 252, and there are 66 images in the test dataset; all images are adjusted to a uniform size before training, and no data enhancement is performed during training; due to the small number of training sets compared with TN3K, TransUNet and Swin UNet perform poorly, and their segmentation performance indicators are inferior to traditional neural networks; the diffusion segmentation model of the present invention achieves the best results, with IoU and DSC values ​​of 71.91% and 81.26%, respectively, which are 1.24% and 1.62% higher than the original diffusion segmentation model (SegDiff), respectively.

[0082] The number of images in private dataset 2 is not much different from that in dataset 1. In private dataset 2, the training set consists of 264 images and the test set has 66 images. However, the images in the dataset come from multiple devices, most of which are noisy. Therefore, the segmentation performance of all models has declined. The DSC index of Swin-UNet has decreased by 10.79% compared with dataset 1, which is the largest decrease among all models. For the segmentation model, the feature information itself will be noisy during the training process. Therefore, the impact on the diffusion model is small, and the DSC index has only decreased by 2.67%. Compared with the original diffusion segmentation model (SegDiff), the model of the present invention improves IoU and DSC by 1.43% and 2.52%, respectively.

[0083] In addition, the accuracy, Dice Similarity Coefficient (DSC) and Intersection over Union (IoU) were used to quantitatively evaluate the model of the present invention and other models, and the performance of the loss function of the model of the present invention and the five models during the training process was compared. Figure 8 As shown, the model of the present invention Figure 8 (f) and the existing model Figure 8(a)-(e), that is, the changes in the loss function of FCN, Attention U-Net, TransUNet, Swin-UNet, and SegDiff during the training process. It can be seen that whether it is a traditional neural network or a Transformer model, when the amount of data is small, the loss function will have obvious fluctuations during the training process. Thyroid ultrasound images are affected by the patient's condition, the severity of the disease, the influence of the instrument, etc., and the differences are very large. Even the thyroid images of the same person will have obvious differences. Therefore, these models are difficult to fit in such highly variable data; however, the diffusion model does not need to fit these data, and only needs to learn to predict the noise map of the next stage to achieve good segmentation performance; therefore, during the training process, the loss of SegDiff and the model of the present invention drops rapidly without obvious fluctuations, but this does not mean that the model has not learned. In the experiment, it is found that after the loss value reaches the lowest point, it will continue to change as the training progresses, but the change in the value is very small. The experiment found that although the loss value changes very little, the segmentation performance is indeed improving.

[0084] Ablation experiment:

[0085] Fig. 9 The comparison effect between the model adding module of the present invention and the SegDiff model is shown. The noise image predicted by the output model shows that the original SegDiff noise map is in the 3rd and 7th columns, and the edges of the predicted noise images are irregular. This is also the reason for the mask edge problem obtained after inverse diffusion. The reason may be that the tissue boundaries of the ultrasound image are not obvious. The 4th and 8th columns are the diffusion segmentation models after the pyramid edge weighting module is introduced. The boundary information is learned separately through the pyramid module, and then the learned features are weighted to the main feature map to supplement the boundary information, so that the boundary area of ​​the noise image predicted by the model is relatively smooth. It can be seen from the figure that the prediction of the noise image by the model of the present invention is improved, whether it is a large target or a small target, a single target or multiple targets.

[0086] Table 2 Ablation experiments of diffusion model training steps

[0087]

[0088] Table 2 shows the segmentation performance indicators obtained by the diffusion segmentation model with different step numbers. Three key time intervals are analyzed. Between 60,000 steps and 100,000 steps, the segmentation performance of the model of the present invention is significantly improved; in SegDiff, IoU and DSC are improved by 6.95% and 16.52%, respectively; in the model of the present invention, IoU and DSC are improved by 8.85% and 16.69%, respectively; however, between 100,000 steps and 140,000 steps, the segmentation performance indicators are almost not increased; SegDiff only increases IoU and DSC by 0.27% and 0.63%, respectively; in the model of the present invention, it is only increased by 0.11% and 0.05%; although the improvement of the indicators is small, the model is still learning and optimizing, and the segmentation performance of 180,000 steps and 200,000 steps is also tested, and the results are also slightly improved.

[0089] The segmentation performance indicators obtained by the diffusion segmentation model under different training sample sizes are given, as shown in Table 3;

[0090] Table 3 Comparison of the impact of different data volumes on segmentation performance

[0091]

[0092] The performance of the diffusion segmentation model on a small sample dataset was verified using the public dataset TN3K. The original TN3K dataset consisted of 3493 images, which were divided into training and test sets in a ratio of 8:2. The final training samples were 2879. In subsequent experiments, the number of training samples was continuously reduced until it was reduced to 1 / 5 of the original number. Although the performance of the indicators declined, the gap between the partial data and the complete data indicators was between 2.05% and 2.93%.

[0093] The present invention proposes a PEW-SegDiff model for thyroid ultrasound image segmentation. By adding a pyramid boundary learning module to extract edge features, and calculating weights at multiple scales, the final weights are added to the main path feature map. It mainly solves the problem of uneven tissue boundaries caused by diffusion segmentation, and has a good performance improvement when facing nodule segmentation targets with irregular shapes and different sizes. The experiment verified that the diffusion model can still achieve good segmentation performance indicators when facing a small amount of data.

[0094] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules, characterized in that: The following steps are involved: Step 1: Obtain ultrasound images of thyroid nodules; Step 2: Add Gaussian noise to the real mask image through time during the forward diffusion process to obtain a noisy label; In the backward diffusion process, the input image and the true mask image after forward diffusion are added to obtain the noise label feature map; The PEW-SegDiff model is trained using the noise label feature map and the input map to obtain the predicted initial image; Compute the loss between the predicted image and the back-diffused image of the true mask.

2. The feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules according to claim 1, characterized in that: The PEW-SegDiff model includes: a pyramid edge weighting module and a denoising network; wherein the pyramid edge weighting module is a dual-channel, the first channel includes: the feature map is resized and input into the feature pyramid module, and then group normalized to obtain the first channel feature map; the second channel includes: the feature map is subjected to double convolution and group normalization to obtain the second channel feature; the first channel and the second channel features are unioned and input into the multi-layer weighting module, and then element-wise multiplication is performed with the second channel feature map through an activation function.

3. The feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules according to claim 2, characterized in that: The double convolution consists of a 3×3 convolution and a 1×1 convolution, and a time vector is added between the two convolution blocks.

4. The feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules according to claim 3, characterized in that: The multi-layer weighted module includes: after the feature maps are input into three parallel convolutions with convolution kernels of 1x1, 3x3, and 3x3, they are combined and resized to obtain weight coefficients.

5. The feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules according to claim 2, characterized in that: The ASPP layer in the feature pyramid module is improved to obtain the AT-ASPP layer. The AT-ASPP layer includes: after 1x1 convolution and addition operation with the time vector, 1x1 convolution, 3x3rate=18 convolution, 3x3rate=12 convolution, 3x3rate=12 convolution are input respectively, and then the convolution is combined and input into the S-Att layer.

6. The feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules according to claim 1, characterized in that: The formula for the loss between the predicted image and the back-diffused image of the true mask is: L total =L dice +αL U +βL edge (7) Among them, L dice To calculate the Dice loss of the fused image, α and β are different weights, and L U is the MSE loss of the main path, L edge is the MSE loss of the edge path.

7. The feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules according to claim 6, characterized in that: L U The formula is: Among them, M pred-U is the prediction result of the main path, represents the MSE loss function, M GT is the time X obtained by back diffusion t-1 The segmentation mask of the ground-truth mask.

8. The feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules according to claim 6, characterized in that: L edge The formula is: Among them, M pred-edge It is the prediction result of edge branch prediction.

9. The characteristic pyramid edge-weighted diffusion segmentation system for ultrasound thyroid nodules is characterized by: include: a memory for storing instructions executable by a processor; A processor, configured to execute instructions to implement the feature pyramid edge weighted diffusion segmentation method for ultrasonic thyroid nodules according to any one of claims 1 to 8.

10. A computer readable medium storing computer program code, characterized in that: When the computer program code is executed by a processor, the feature pyramid edge-weighted diffusion segmentation method for ultrasonic thyroid nodules is implemented according to any one of claims 1 to 8.

Images