A dermoscope image segmentation method based on diffusion blur learning

Abstract

The application provides a dermoscope image segmentation method based on diffusion blur learning, and belongs to the technical field of medical image intelligent segmentation. The technical problems of low segmentation accuracy caused by the fuzzy boundary and region of the segmentation target of the medical image are solved. The technical scheme comprises the following steps: S1, data preprocessing is performed on image data; S2, Gaussian noise is iteratively diffused into the skin image; S3, a feature encoder is used to extract semantic information from the input original skin image; S4, the multi-scale features obtained by the feature encoder are subjected to fuzzy rule processing; S5, an iterative denoising operation is performed on the pure noise skin image; and S6, a fusion module based on the image clarity degree is designed to fuse multiple predictions in the denoising process. The application has the beneficial effects of high segmentation accuracy, providing decision support for the segmentation of melanoma in skin images, and providing convenience for the diagnosis of doctors.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical image processing technology, and in particular to a skin image segmentation method based on diffusion fuzzy learning. Background Technology

[0002] Melanoma is a widespread skin cancer worldwide, and its malignant transformation can lead to serious consequences, even death. Because malignant melanomas are more likely to spread to surrounding tissues, early detection and diagnosis are crucial and can significantly improve patient survival rates. Dermoscopy is an effective imaging method for diagnosing skin diseases, but even with this tool, the accuracy of melanoma diagnosis remains limited for inexperienced physicians. Therefore, accurate melanoma segmentation of skin images can provide physicians with more information and support, helping them to more accurately identify and diagnose melanoma, thereby improving patient treatment outcomes and survival rates.

[0003] In the field of medical image processing, denoising has always been a fundamental and crucial problem. Traditional denoising methods typically rely on filters or transform domain processing, such as wavelet transform, mean filtering, and Gaussian filtering. However, these methods often struggle to effectively handle complex noise while preserving image details. In recent years, with the rapid development of deep learning technology, a new image processing method—the denoising diffusion model—has gradually gained widespread attention. The basic principle of the denoising diffusion model is to utilize a deep neural network model, typically the U-Net network model, to map noisy input images to a corresponding clean image space through training. Compared to traditional medical image segmentation methods, the denoising diffusion model possesses stronger nonlinear modeling capabilities and superior generalization performance, enabling it to more effectively capture complex structural and detailed information in images. Despite the significant success of the denoising diffusion model in image segmentation, some challenges and unresolved issues remain in the segmentation of melanoma in skin images. Summary of the Invention

[0004] This invention proposes a dermoscopic image segmentation method based on diffusion fuzzy learning. Considering the uncertainty of segmentation boundaries and the prevalence of regional fuzziness in melanoma in skin images, fuzzy learning theory is added to the denoising diffusion model and the prediction results of each step in the denoising process are fused to more effectively handle the situation of boundary fuzziness and uncertainty, and also enhance the accuracy and robustness of dermoscopic image segmentation.

[0005] This invention is achieved through the following measures: a dermoscopic image segmentation method based on diffusion fuzzy learning, comprising the following steps:

[0006] S1: Read the skin image dataset and preprocess the images. Convert each image to RGB mode, convert the corresponding label to grayscale mode, and crop the skin image and label to a size of 256 pixels in both height and width.

[0007] S2: Iteratively diffuse the noise image into the input image. In the forward process of the diffusion model, an initial noise image is generated from a Gaussian distribution. This initial noise image is a pure noise image that follows a Gaussian distribution. Through multiple iterations, the noise image is diffused into the skin image, so that the noise in the skin image gradually increases, thereby simulating the gradually increasing noise level. Finally, a pure noise skin image is obtained.

[0008] S3: A feature encoder was designed to extract features from the input raw skin image. The feature encoder consists of several convolutional modules. The feature maps obtained by the convolutional modules are saved as the input of the subsequent fuzzy learning module.

[0009] S4: The multi-scale feature maps obtained by the feature encoder are processed by the fuzzy learning module (FLM) for fuzzy rule processing. The feature maps processed by the fuzzy learning module are then saved for fusion with the features of each layer of the encoder of the U-Net++ network in the subsequent denoising process.

[0010] S5: Through the denoising process of the diffusion model, based on the multi-scale semantic information extracted by the feature encoder and fuzzy learning module, the pure noisy skin image with added Gaussian noise is subjected to multiple iterations of denoising operation.

[0011] S6: Design a fusion module based on image sharpness to fuse multiple predictions from the denoising process in order to obtain more robust segmentation results during the testing phase.

[0012] As a further optimization of the dermoscopic image segmentation method based on diffusion fuzzy learning provided by the present invention, the specific steps of step S2 are as follows:

[0013] Step S2.1: Given an initial skin image x0∈R H×W×C Here, H, W, and C represent the height, width, and number of channels of the original skin image, respectively. We progressively add Gaussian noise to the original skin image, maintaining a relatively uniform noise diffusion amplitude at each time step t during the noise addition process. The skin image initially has a very uniform distribution, and adding some noise can alter the original distribution. However, in later stages, more noise needs to be added to ensure a consistent noise diffusion amplitude. Therefore, we set β... t The degree of noise addition is controlled by ∈(0,1), i.e., β. t The value will gradually increase; through a series of noise-adding processes, corresponding noisy images x1,...,x are generated.T , in the case of x t-1 Add noise to x t During the process, the standard deviation / variance of the noise is a fixed value β within the interval (0,1). T The mean is determined by a fixed value β. T and the image data x at the current moment t-1 This is determined by the formula. The noise addition process described above can be written as:

[0014]

[0015]

[0016] Where, β t ∈(0,1) is the variance hyperparameter of a Gaussian distribution, and satisfies β1<β2<…<β T By x t-1 Get x t The process satisfies the distribution Therefore, the noise is only determined by β. t and x t-1 Determining this is a fixed-value process, not a learnable one. Therefore, given x0 and the fixed values ​​β1,...,β at each time step determined in advance... T This allows us to derive the noisy data x1,...,x at any time step. T ;

[0017] Step S2.2: As the time step increases, the original skin image will gradually lose its features. When the time step T→∞, x T It approximates an isotropic Gaussian distribution. Visually, this means that after adding noise to the original skin image for 1000 time steps, the skin image becomes a purely noisy image. Furthermore, during the noise addition process, we do not need to iterate from x0 to x step by step. t Instead, it can be derived from x0 and a fixed value sequence. Direct calculation yields:

[0018]

[0019] in, Due to β t ∈(0,1) and β1<…<β T , and α t =1-β t Therefore α t ∈(0,1) and have α1>…>α T .because As T→∞ as well as at this time

[0020] As a further optimization of the dermoscopic image segmentation method based on diffusion fuzzy learning provided by the present invention, the specific steps of step S3 are as follows:

[0021] Step S3.1: Given the original skin image I∈R H×W×C As conditions at each time step of the denoising diffusion model, H, W, and C represent the height, width, and number of channels of the skin image, respectively. At time step t, to better incorporate the original skin image as a conditional embedding, we use the skin image I as input to the feature encoder to obtain multi-scale conditional features. k is the layer index of the convolutional module. The feature encoder consists of four convolutional modules, each including a convolutional layer, a pooling layer, and an activation function. A small convolutional kernel is used to capture local features; max pooling is selected in the pooling layer to reduce the size of the feature map; the activation function is the ReLU activation function.

[0022] Step S3.2: When the original skin image passes through a convolutional module, it first goes through a 3x3 convolutional layer to obtain a feature map. Next, to increase the network's non-linear expressive power, a ReLU activation function is applied after the convolutional operation to introduce non-linearity. Finally, max pooling is applied to reduce the size of the feature map while retaining the main features. After processing by the convolutional module, each module downsamples the feature map by a factor of two, sequentially obtaining the corresponding features. These features are preserved and, after fuzzy rule processing, will be fused with the feature maps of the encoder in the subsequent U-Net++ network.

[0023] As a further optimization of the dermoscopic image segmentation method based on diffusion fuzzy learning provided by the present invention, the specific steps of step S4 are as follows:

[0024] Step S4.1: Convert the multi-scale features processed by the convolutional module of the feature encoder. The inputs are fed into the Fuzzy Learning Module (FLM). The fuzzy modules in the connection paths of the feature encoder and the U-Net++ network's encoder have the same structure, differing only in the size of the input and output feature maps. For each channel's feature, a fuzzy membership function is applied to each feature point in the feature map. It's important to note that the fuzzy membership function remains constant across each channel of each feature map, but can change across different channels. Each membership function assigns a membership degree to each pixel, describing its similarity to other pixels. A Gaussian function is used as the membership function in the membership function layer. The output of the first convolutional module in the feature encoder mentioned in S3 is used as an example. For example, it can be expressed by the following formula:

[0025]

[0026] in, It is the output of the i-th membership function, where t represents the t-th pixel of the feature map and c represents the c-th channel of the feature map. It is the t-th pixel that is used as input in the feature map of the c-th channel, u i,c and σ i ,c is the mean and standard deviation of the i-th Gaussian membership function in the c-th channel. Note that the parameter u... i,c and σ i The value of 'c' remains constant within the same channel but may differ between different channels. This is because feature points in the same channel are extracted by the same convolutional kernel, thus possessing similar features; while feature points in different channels are extracted by different convolutional kernels, resulting in different features. Therefore, the number of parameters in each fuzzy learning module can be calculated as M×C×2, which is comparable to the number of parameters in a single convolutional layer. Furthermore, the number of membership functions, M, is the only hyperparameter that needs to be preset to implement the fuzzy learning module. By applying AND fuzzy logic to all membership functions of the feature points, the final fuzzy feature information of the input feature map for the c-th channel is obtained using the following method.

[0027]

[0028] in, It is the output of the i-th membership function. Formula (5) is based on the output of each membership function. Finally, fuzzy feature information processed by the fuzzy learning module is obtained. Using the method described above, we can obtain the blurry feature information of each pixel in each channel. Therefore, we obtain the output blurriness tensor of each blur learning module. k is the output of the k-th fuzzy learning module.

[0029] Step S4.2: Finally, the output of the fuzzy learning module is the fuzziness tensor. Its size is related to the corresponding input feature map tensor. Same, that is Before integration, the output feature map and fuzzy tensor of the feature encoder are processed by batch normalization (BN) to constrain their dynamic range. Then, a simple addition operation is used to integrate the fuzzy logic information.

[0030] As a further optimization of the dermoscopic image segmentation method based on diffusion fuzzy learning provided by the present invention, the specific steps of step S5 are as follows:

[0031] Step S5.1: In the reverse process, the neural network (U-Net++ network) is trained to remove added Gaussian noise to make the purely noisy image x T Restore to the original image x0. This process can be represented as:

[0032]

[0033] Where θ is the model parameter, p θ This is a deep learning model. The reverse process mainly aims to obtain a clear segmentation result by removing noise. To better learn the denoising process, we employ a classic neural network (U-Net++ network) and a feature encoder for feature extraction from the original skin image. First, the purely noisy image x obtained through the forward process of the diffusion model in S2 is... T ∈R H×W×C The input is fed into the encoder of the U-Net++ network. The U-Net++ network contains four downsampling blocks and four upsampling blocks. Each downsampling block consists of a convolutional layer, a pooling layer, an activation function, and a batch normalization layer. The convolutional kernel size is 3x3, with a stride of 1 and padding of 1. Max pooling is used, with a pooling size of 2x2 and a stride of 2. ReLU activation is employed. Each upsampling block consists of a transposed convolutional layer, skip connections, an activation function, and a batch normalization layer. The convolutional kernel size is 3x3, with a stride of 1 and padding of 1. ReLU activation is employed.

[0034] Step S5.2: The pure noisy skin image with added Gaussian noise is first used as the input to the U-Net++ network, and then processed through four downsampling blocks to obtain the corresponding multi-scale features. Where t represents the t-th time step in the denoising process, and k represents the index of the downsampling block. For each downsampling block, the image features are first extracted through a convolutional layer. An activation function is then applied after the convolutional layer to introduce non-linearity. Max pooling is then used to reduce the size of the feature map while preserving the main features. Finally, batch normalization is applied to accelerate network training and improve model stability. Each downsampling block downsamples the feature map by a factor of two. The outputs of the 1st to 4th downsampling blocks are as follows: because and the output features of the fuzzy learning module Since they contain the same number and size of features, features at corresponding scales are added together to obtain fused features.

[0035] Step S5.3: Input the output features of the encoder of the U-Net++ network into the decoder for upsampling. The decoder consists of four upsampling blocks, and the skip path consists of dense convolutional blocks with multiple convolutional layers, where each convolutional layer is preceded by a cascaded layer that fuses the output of the previous convolutional layer of the same dense block with the corresponding upsampled output of the lower dense block. Specifically, let f i,j Indicates the F of each layer of the U-Net network encoder i,j The output of f, where i is the index of the downsampled block along the encoder, and j is the index of the convolutional layer of the dense block along the jump path. i,j The feature map stack calculation is as follows:

[0036]

[0037] Here, function A(·) represents a convolution operation, followed by the application of an activation function; [] represents a concatenated layer; and U represents an upsampling layer. Essentially, a node at level j=0 receives only one input from the layer above the U-Net encoder; a node at level j=1 receives two inputs, both from downsampled blocks of the encoder, but at two consecutive levels; nodes at levels j>1 receive j+1 inputs, where j inputs are the outputs of the first j nodes in the same jump path, and the last input is the upsampled output of the next jump path. All previous feature maps accumulate and reach the current node because dense convolutional blocks are used along each jump path.

[0038] Step S5.4: After processing the dense convolutional blocks in a series of skip paths, the final features of each block are fused with the features in the upsampled block to obtain the final prediction result x. T-1 Subsequently, based on the process described above, regarding x... T-1 Perform iterative denoising operations until a clear segmented image x0 is obtained.

[0039] As a further optimization of the dermoscopic image segmentation method based on diffusion fuzzy learning provided by the present invention, the specific steps of step S6 are as follows:

[0040] Step S6.1: Image sharpness is an indicator of the richness or variability of information in an image. We use the image variance as an indicator of sharpness. A larger variance indicates more drastic changes in pixel values ​​within the image, generally meaning a sharper image. The output prediction for each time step has a corresponding variance. Since i = 1, ..., T, image variance is used as a measure of sharpness. The weights are calculated by summing the variances of the predictions at all time steps to obtain the total variance, and then dividing the variance of each prediction by the total variance to obtain the weight of each prediction. A larger weight indicates higher image sharpness and a greater contribution to the fusion result.

[0041] Step S6.2: Since the weights are proportional to the sharpness, we can simply normalize the sharpness to a probability distribution. The total sharpness is... The weight of each image is the ratio of its sharpness to the total sharpness, that is:

[0042]

[0043] Among them, w i The weights for each image, P i The sharpness ratio for each image, i = 1, ..., T, represents the skin image x excluding pure noise. T All other prediction results generated by the denoising process will have corresponding fusion weights calculated, and the higher the image resolution, the greater its weight and the greater its contribution to the fusion result.

[0044] Step S6.2: Using the fusion weights obtained above, the prediction results generated at each time step of the denoising process are weighted to obtain the final fusion result Y, which is used as the output of our network.

[0045]

[0046] Where, x i w represents the prediction result of the denoising process at the i-th time step. i These are the fusion weights for the corresponding time steps. The fused result Y is normalized to limit the pixels to a reasonable range.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] 1. The introduced denoising diffusion model eliminates the need for manually designing feature extractors; instead, it automatically learns image features through deep learning. This makes the model more flexible, adaptable to different types and sizes of images, without requiring manual adjustments.

[0049] 2. The introduced U-Net++ is a classic architecture widely used in medical image segmentation. It has an encoder-decoder structure and skip connections, which can effectively learn features at different scales and improve the performance of the model.

[0050] 3. To overcome the problems of uncertain and ambiguous segmentation boundaries in medical images, this paper introduces a parallel fuzzy learning module to enhance the model's ability to model uncertain and ambiguous segmentation boundaries;

[0051] 4. To improve the accuracy and robustness of the model's prediction results, this paper proposes a fusion module based on image sharpness to optimize the segmentation results. Attached Figure Description

[0052] The technical solution and its beneficial effects of the present invention will become apparent from the following detailed description of specific embodiments of the invention, in conjunction with the accompanying drawings.

[0053] Figure 1 The diagram shows a flowchart of a dermoscopic image segmentation method based on diffusion fuzzy learning according to the present invention.

[0054] Figure 2 The figure shown is an overall framework diagram of a dermoscopic image segmentation method based on diffusion fuzzy learning according to the present invention.

[0055] Figure 3 The diagram shows a fuzzy learning module of a dermoscopic image segmentation method based on diffusion fuzzy learning according to the present invention.

[0056] Figure 4 The diagram shown is a U-Net++ network architecture diagram of a dermoscopic image segmentation method based on diffusion fuzzy learning according to the present invention.

[0057] Figure 5 The image shown is the result of segmenting a skin image dataset using a dermoscopic image segmentation method based on diffusion fuzzy learning according to the present invention.

[0058] Figure 6 The image shown is the result of segmenting the Brats2020 dataset using a dermoscopic image segmentation method based on diffusion fuzzy learning according to the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. Of course, the examples described with reference to the accompanying drawings are only for explaining the invention and should not be construed as limiting it.

[0060] Example 1

[0061] See Figures 1 to 5 This embodiment provides a technical solution for dermoscopic image segmentation based on diffusion fuzzy learning. Taking the first image in the dermoscopic image dataset as an example, the method from loading data to obtaining the segmentation result includes the following steps:

[0062] S1: Read the skin image dataset and preprocess the images. Convert each image to RGB mode, convert the corresponding label to grayscale mode, and crop the skin image and label to a size of 256 pixels in both height and width.

[0063] S2: Iteratively diffuse the noise image into the input image. In the forward process of the diffusion model, an initial noise image is generated from a Gaussian distribution. This initial noise image is a pure noise image that follows a Gaussian distribution. Through multiple iterations, the noise image is diffused into the skin image, so that the noise in the skin image gradually increases, thereby simulating the gradually increasing noise level. Finally, a pure noise skin image is obtained.

[0064] S3: A feature encoder was designed to extract features from the input raw skin image. The feature encoder consists of several convolutional modules. The feature maps obtained by the convolutional modules are saved as the input of the subsequent fuzzy learning module.

[0065] S4: The multi-scale feature maps obtained by the feature encoder are processed by the fuzzy learning module (FLM) for fuzzy rule processing. The feature maps processed by the fuzzy learning module are then saved for fusion with the features of each layer of the encoder of the U-Net++ network in the subsequent denoising process.

[0066] S5: Through the denoising process of the diffusion model, based on the multi-scale semantic information extracted by the feature encoder and fuzzy learning module, the pure noisy skin image with added Gaussian noise is subjected to multiple iterations of denoising operation.

[0067] S6: Design a fusion module based on image sharpness to fuse multiple predictions from the denoising process in order to obtain more robust segmentation results during the testing phase.

[0068] Specifically, the steps of step S2 are as follows:

[0069] Step S2.1: Given an initial skin image x0∈R H×W×C Here, H, W, and C represent the height, width, and number of channels of the original skin image, respectively. We progressively add Gaussian noise to the original skin image, maintaining a relatively uniform noise diffusion amplitude at each time step t during the noise addition process. The skin image initially has a very uniform distribution, and adding some noise can alter the original distribution. However, in later stages, more noise needs to be added to ensure a consistent noise diffusion amplitude. Therefore, we need to set β... t The degree of noise addition is controlled by ∈(0,1), i.e., β. t The value will gradually increase. Through a series of noise-adding processes, corresponding noisy images x1,...,x are generated. T In the case of x t-1Add noise to x t During the process, the standard deviation / variance of the noise is a fixed value β within the interval (0,1). T The mean is determined by a fixed value β. T and the image data x at the current moment t-1 This is determined by the formula. The noise addition process described above can be written as:

[0070]

[0071]

[0072] Where, β t ∈(0,1) is the variance hyperparameter of a Gaussian distribution, and satisfies β1<β2<…<β T By x t-1 Get x t The process satisfies the distribution Therefore, the noise is only determined by β. t and x t-1 Determining this is a fixed-value process, not a learnable one. Therefore, given x0 and the fixed values ​​β1,...,β at each time step determined in advance... T This allows us to derive the noisy data x1,...,x at any time step. T ;

[0073] Step S2.2: As the time step increases, the original skin image will gradually lose its features. When the time step T→∞, x T It approximates an isotropic Gaussian distribution. Visually, this means that after adding noise to the original skin image for 1000 time steps, the skin image becomes a purely noisy image. Furthermore, during the noise addition process, we do not need to iterate from x0 to x step by step. t Instead, it can be derived from x0 and a fixed value sequence. Direct calculation yields:

[0074]

[0075] in, Due to β t ∈(0,1) and β1<…<β T , and α t =1-β t Therefore α t ∈(0,1) and have α1>…>α T .because As T→∞ as well as at this time

[0076] Specifically, the steps of step S3 are as follows:

[0077] Step S3.1: Given the original skin image I∈R H×W×C As conditions at each time step of the denoising diffusion model, H, W, and C represent the height, width, and number of channels of the skin image, respectively. At time step t, to better incorporate the original skin image as a conditional embedding, we use the skin image I as input to the feature encoder to obtain multi-scale conditional features. k is the layer index of the convolutional module. The feature encoder consists of four convolutional modules, each including a convolutional layer, a pooling layer, and an activation function. A small convolutional kernel is used to capture local features; max pooling is selected in the pooling layer to reduce the size of the feature map; the activation function is the ReLU activation function.

[0078] Step S3.2: When the original skin image passes through a convolutional module, it first goes through a 3x3 convolutional layer to obtain a feature map. Next, to increase the network's non-linear expressive power, a ReLU activation function is applied after the convolutional operation to introduce non-linearity. Finally, max pooling is applied to reduce the size of the feature map while retaining the main features. After processing by the convolutional module, each module downsamples the feature map by a factor of two, sequentially obtaining the corresponding features. These features are preserved and, after fuzzy rule processing, will be fused with the feature maps of the encoder in the subsequent U-Net++ network.

[0079] Specifically, the steps of step S4 are as follows:

[0080] Step S4.1: Convert the multi-scale features processed by the convolutional module of the feature encoder. The inputs are fed into the Fuzzy Learning Module (FLM). The fuzzy modules in the connection paths of the feature encoder and the U-Net++ network's encoder have the same structure, differing only in the size of the input and output feature maps. For each channel's feature, a fuzzy membership function is applied to each feature point in the feature map. It's important to note that the fuzzy membership function remains constant across each channel of each feature map, but can change across different channels. Each membership function assigns a membership degree to each pixel, describing its similarity to other pixels. A Gaussian function is used as the membership function in the membership function layer. The output of the first convolutional module in the feature encoder mentioned in S3 is used as an example. For example, it can be expressed by the following formula:

[0081]

[0082] in, It is the output of the i-th membership function, where t represents the t-th pixel of the feature map and c represents the c-th channel of the feature map. It is the t-th pixel that is used as input in the feature map of the c-th channel, u i,c and σ i ,c is the mean and standard deviation of the i-th Gaussian membership function in the c-th channel. Note that the parameter u... i,c and σ i The value of 'c' remains constant within the same channel but may differ between different channels. This is because feature points in the same channel are extracted by the same convolutional kernel, thus possessing similar features; while feature points in different channels are extracted by different convolutional kernels, resulting in different features. Therefore, the number of parameters in each fuzzy learning module can be calculated as M×C×2, which is comparable to the number of parameters in a single convolutional layer. Furthermore, the number of membership functions, M, is the only hyperparameter that needs to be preset to implement the fuzzy learning module. By applying AND fuzzy logic to all membership functions of the feature points, the final fuzzy feature information of the input feature map for the c-th channel is obtained using the following method.

[0083]

[0084] in, It is the output of the i-th membership function. Formula (5) is based on the output of each membership function. Finally, fuzzy feature information processed by the fuzzy learning module is obtained. Using the method described above, we can obtain the blurry feature information of each pixel in each channel. Therefore, we can obtain the output blur tensor of each blur learning module. k is the output of the k-th fuzzy learning module.

[0085] Step S4.2: Finally, the output of the fuzzy learning module is the fuzziness tensor. Its size is related to the corresponding input feature map tensor. Same, that is Before integration, the output feature map and fuzzy tensor of the feature encoder are processed by batch normalization (BN) to constrain their dynamic range. Then, a simple addition operation is used to integrate the fuzzy logic information.

[0086] Specifically, the steps of step S5 are as follows:

[0087] Step S5.1: In the reverse process, the neural network (U-Net++ network) is trained to remove added Gaussian noise to make the purely noisy image x T Restore to the original image x0. This process can be represented as:

[0088]

[0089] Where θ is the model parameter, p θ This is a deep learning model. The reverse process mainly aims to obtain a clear segmentation result by removing noise. To better learn the denoising process, we employ a classic neural network (U-Net++ network) and a feature encoder for feature extraction from the original skin image. First, the purely noisy image x obtained through the forward process of the diffusion model in S2 is... T ∈R H×W×C The input is fed into the encoder of the U-Net++ network. The U-Net++ network contains four downsampling blocks and four upsampling blocks. Each downsampling block consists of a convolutional layer, a pooling layer, an activation function, and a batch normalization layer. The convolutional kernel size is 3x3, with a stride of 1 and padding of 1. Max pooling is used, with a pooling size of 2x2 and a stride of 2. ReLU activation is employed. Each upsampling block consists of a transposed convolutional layer, skip connections, an activation function, and a batch normalization layer. The convolutional kernel size is 3x3, with a stride of 1 and padding of 1. ReLU activation is employed.

[0090] Step S5.2: The pure noisy skin image with added Gaussian noise is first used as the input to the U-Net++ network, and then processed through four downsampling blocks to obtain the corresponding multi-scale features. Where t represents the t-th time step in the denoising process, and k represents the index of the downsampling block. For each downsampling block, the image features are first extracted through a convolutional layer. An activation function is then applied after the convolutional layer to introduce non-linearity. Max pooling is then used to reduce the size of the feature map while preserving the main features. Finally, batch normalization is applied to accelerate network training and improve model stability. Each downsampling block downsamples the feature map by a factor of two. The outputs of the 1st to 4th downsampling blocks are as follows: because and the output features of the fuzzy learning module Since they contain the same number and size of features, features at corresponding scales are added together to obtain fused features.

[0091] Step S5.3: Input the output features of the encoder of the U-Net++ network into the decoder for upsampling. The decoder consists of four upsampling blocks, and the skip path consists of dense convolutional blocks with multiple convolutional layers, where each convolutional layer is preceded by a cascaded layer that fuses the output of the previous convolutional layer of the same dense block with the corresponding upsampled output of the lower dense block. Specifically, let f i,j Indicates the F of each layer of the U-Net network encoderi,j The output of f, where i is the index of the downsampled block along the encoder, and j is the index of the convolutional layer of the dense block along the jump path. i,j The feature map stack calculation is as follows:

[0092]

[0093] Here, function A(·) represents a convolution operation, followed by the application of an activation function; [] represents a concatenated layer; and U represents an upsampling layer. Essentially, a node at level j=0 receives only one input from the layer above the U-Net encoder; a node at level j=1 receives two inputs, both from downsampled blocks of the encoder, but at two consecutive levels; nodes at levels j>1 receive j+1 inputs, where j inputs are the outputs of the first j nodes in the same jump path, and the last input is the upsampled output of the next jump path. All previous feature maps accumulate and reach the current node because dense convolutional blocks are used along each jump path.

[0094] Step S5.4: After processing the dense convolutional blocks in a series of skip paths, the final features of each block are fused with the features in the upsampled block to obtain the final prediction result x. T-1 Subsequently, based on the process described above, regarding x... T-1 Perform iterative denoising operations until a clear segmented image x0 is obtained.

[0095] Specifically, the steps of step S6 are as follows:

[0096] Step S6.1: Image sharpness is an indicator of the richness or variability of information in an image. We use the image variance as an indicator of sharpness. A larger variance indicates more drastic changes in pixel values ​​within the image, generally meaning a sharper image. The output prediction for each time step has a corresponding variance. Since i = 1, ..., T, image variance is used as a measure of sharpness. The weights are calculated by summing the variances of the predictions at all time steps to obtain the total variance, and then dividing the variance of each prediction by the total variance to obtain the weight of each prediction. A larger weight indicates higher image sharpness and a greater contribution to the fusion result.

[0097] Step S6.2: Since the weights are proportional to the sharpness, we can simply normalize the sharpness to a probability distribution. The total sharpness is... The weight of each image is the ratio of its sharpness to the total sharpness, that is:

[0098]

[0099] Among them, wi The weights for each image, P i The sharpness ratio for each image, i = 1, ..., T, represents the skin image x excluding pure noise. T All other prediction results generated by the denoising process will have corresponding fusion weights calculated, and the higher the image resolution, the greater its weight and the greater its contribution to the fusion result.

[0100] Step S6.2: Using the fusion weights obtained above, the prediction results generated at each time step of the denoising process are weighted to obtain the final fusion result Y, which is used as the output of our network.

[0101]

[0102] Where, x i w represents the prediction result of the denoising process at the i-th time step. i These are the fusion weights for the corresponding time steps. The fused result Y is normalized to limit the pixels to a reasonable range.

[0103] like Figure 2 As shown, given a skin image I∈R H×W×C Noise is added to image I to obtain a skin image x with pure Gaussian noise. T ∈R H×W×C .like Figure 2 As shown, skin image I is input into a feature encoder to obtain multi-scale features. The convolutional kernel size is set to x_conv*x_conv, the stride to t_step, and padding is performed. Max pooling is chosen in the pooling layer to reduce the feature map size. The ReLU activation function is used. Each multi-scale feature is then sequentially input into the fuzzy learning module to obtain the corresponding features processed by fuzzy rules. like Figure 3 As shown. Based on this feature, denoising is performed on skin images with added Gaussian noise. First, the feature map to be denoised is used as input to the U-Net++ network model, as shown. Figure 4 As shown, after processing through four downsampling blocks, the result is the same as described above. Features k = 1, ..., 4 have the same number and size, so features at corresponding scales are summed to obtain fused features. These fused features are then input into the decoder of the U-Net network, where semantic information is enriched through skip connections with dense convolutional blocks. Finally, the prediction result for this time step is obtained. Based on the denoising process described above, iterative denoising operations yield the final prediction result x0∈R. H×W×CTo improve the accuracy and robustness of the model's predictions, a fusion module based on image sharpness was designed to fuse the segmentation maps obtained at each time step to obtain the final segmentation result, such as... Figure 5 As shown.

[0104] To verify the accuracy and robustness of our model's segmentation, we trained and tested it on the Brats2020 dataset. First, given a brain MRI scan image I∈R... H×W×C Noise is added to image I to obtain a brain image x with pure Gaussian noise. T ∈R H×W×C Then, the brain image I is input into the feature encoder to obtain multi-scale features. Then, each multi-scale feature is sequentially input into the fuzzy learning module to obtain the corresponding features processed by fuzzy rules. like Figure 3 As shown. Based on this feature, denoising is performed on brain images with added Gaussian noise. First, the feature map to be denoised is used as input to the U-Net++ network model, such as... Figure 4 As shown, after processing through four downsampling blocks, the result is the same as described above. Features k = 1, ..., 4 have the same number and size, so features at corresponding scales are summed to obtain fused features. These fused features are then input into the decoder of the U-Net network, where semantic information is enriched through skip connections with dense convolutional blocks. Finally, the prediction result for this time step is obtained. Based on the denoising process described above, iterative denoising operations yield the final prediction result x0∈R. H×W×C To improve the accuracy and robustness of the model's predictions, a fusion module based on image sharpness was designed to fuse the segmentation maps obtained at each time step to obtain the final segmentation result, such as... Figure 6 As shown.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dermoscopic image segmentation method based on diffusion fuzzy learning, characterized in that, Includes the following steps: S1: Read the skin image dataset and preprocess the images; S2: Iteratively diffuse the noise image into the skin image. In the forward process of the diffusion model, an initial noise image is generated from a Gaussian distribution. This initial noise image is a pure noise image that follows a Gaussian distribution. By iterating multiple times, the noise image is diffused into the skin image, so that the noise in the skin image gradually increases, thereby simulating the gradually increasing noise level. Finally, a pure noise skin image is obtained. S3: A feature encoder was designed to extract features from the input raw skin image. The feature encoder consists of several convolutional modules. The feature maps obtained by the convolutional modules are saved as the input of the subsequent fuzzy learning module. S4: The multi-scale feature maps obtained by the feature encoder are processed by the fuzzy learning module FLM for fuzzy rule processing. Then, the feature maps processed by the fuzzy learning module are saved for fusion with the features of each layer of the encoder of the U-Net++ network in the subsequent denoising process. The steps of step S4 are as follows: Step S4.1: Convert the multi-scale features processed by the convolutional module of the feature encoder. The inputs are fed into the fuzzy learning module (FLM). The fuzzy modules on the connection paths of the feature encoder and the U-Net++ network's encoder have the same structure. For each channel's feature, a fuzzy membership function is applied to each feature point in the feature map. The fuzzy membership function remains unchanged in each channel of each feature map but changes in different channels. Each membership function assigns a membership degree to each pixel to describe its similarity to other pixels. In the membership function layer, a Gaussian function is used as the membership function, based on the output of the first convolutional module in the feature encoder mentioned in step S3. The formula is as follows: ； in, This is the output of the i-th membership function, where t represents the t-th pixel of the feature map, and c represents the c-th channel of the feature map. It is the t-th pixel used as input in the feature map of the c-th channel. and The mean and standard deviation of the i-th Gaussian membership function in the c-th channel are used to obtain the final fuzzy feature information of the input feature map of the c-th channel using the following method. : ； in, It is the output of the i-th membership function. Formula (5) is based on the output of each membership function. Finally, fuzzy feature information processed by the fuzzy learning module is obtained. ; Step S4.2: The output of the fuzzy learning module is a fuzziness tensor. Its size is related to the corresponding input feature map tensor. Same, that is , , , , Before integration, the output feature map and fuzzy tensor of the feature encoder are processed by batch normalization (BN) to constrain their dynamic range, and then a simple addition operation is used to integrate the fuzzy logic information. S5: Through the denoising process of the diffusion model, based on the multi-scale semantic information extracted by the feature encoder and fuzzy learning module, the pure noisy skin image with added Gaussian noise is subjected to multiple iterations of denoising operation. S6: Design a fusion module based on image sharpness to fuse multiple predictions from the denoising process in order to obtain the expected segmentation results during the testing phase; The steps of step S6 are as follows: Step S6.1: The output prediction for each time step has a corresponding variance. Image variance is used as a measure of sharpness. The weight is calculated by adding the variances of the prediction results at all time steps to obtain the total variance, and then dividing the variance of each prediction result by the total variance to obtain the weight of each prediction result. Step S6.2: Since the weights are proportional to the sharpness, the sharpness is normalized to a probability distribution, and the total sharpness is... The weight of each image is the ratio of its sharpness to the total sharpness, that is: ； in, Weights for each image, The resolution ratio for each image, This indicates skin images other than pure noise. ; Step S6.3: Using the fusion weights obtained above, the prediction results generated at each time step of the denoising process are weighted to obtain the final fusion result Y, which is used as the network output: ； in, This represents the prediction result of the denoising process at the i-th time step. To determine the fusion weights for the corresponding time steps, the fused result Y is normalized to limit the pixels to a reasonable range.

2. The dermoscopic image segmentation method based on diffusion fuzzy learning according to claim 1, characterized in that, The steps of step S2 are as follows: Step S2.1: Given an initial skin image Where H, W, and C represent the height, width, and number of channels of the original skin image, respectively; [Settings...] To control the degree of noise addition, i.e. As the value gradually increases, a series of noise-adding processes are used to generate a corresponding noisy image. , in the Add noise During the process, the standard deviation or variance of the noise is a fixed value within the interval (0,1). The mean is determined by a fixed value. Image data at the current moment Therefore, the noise addition process described above can be written as a formula: ； ； in, Let be the variance hyperparameter of a Gaussian distribution, and satisfy . ,Depend on get The process satisfies the distribution Noise is caused by and To determine; Step S2.2: When time step hour, Approaching an isotropic Gaussian distribution, after adding noise to the original skin image for 1000 time steps, the skin image becomes a purely noisy image. During the process of adding noise, from... and fixed value sequence The calculation yielded: ； in, ,because and ,and ,therefore And there are ,because ,when hour, as well as ,at this time .

3. The dermoscopic image segmentation method based on diffusion fuzzy learning according to claim 1, characterized in that, The steps of step S3 are as follows: Step S3.1: Given the original skin image As conditions for each time step of the denoising diffusion model, H, W, and C represent the height, width, and number of channels of the skin image, respectively. At time step t, the skin image I is used as the input to the feature encoder to obtain multi-scale conditional features. k is the layer index of the convolutional module. The feature encoder consists of four convolutional modules, each of which includes a convolutional layer, a pooling layer, and an activation function. A small convolutional kernel is used to capture local features. The activation function is the ReLU activation function. Step S3.2: When the original skin image passes through a convolutional module, it first goes through a 3*3 convolutional layer to obtain a feature map. Then, after the convolution operation in the convolutional layer, a ReLU activation function is applied to introduce non-linearity, and max pooling is applied to retain the main features. After processing by the convolutional module, each module downsamples the feature map by a factor of two, thus obtaining the corresponding features. , , , , These features are preserved and, after fuzzy rule processing, will be fused with the feature maps of the encoder in the subsequent U-Net++ network.

4. The dermoscopic image segmentation method based on diffusion fuzzy learning according to claim 1, characterized in that, The steps of step S5 are as follows: Step S5.1: In the reverse process, the U-Net++ neural network is trained to remove added Gaussian noise from the pure noise image. Restore to the original image This process is represented as: ； in, For model parameters, For deep learning models, the reverse process removes noise to obtain clear segmentation results; Step S5.2: The pure noisy skin image with added Gaussian noise is first used as input to the U-Net++ network, and then processed through four downsampling blocks to obtain the corresponding multi-scale features. Where t represents the t-th time step of the denoising process, and k represents the index of the downsampling block; Step S5.3: Input the output features of the encoder of the U-Net++ network into the decoder for upsampling. The decoder consists of four upsampling blocks and the skip path consists of dense convolutional blocks with multiple convolutional layers. Each convolutional layer is preceded by a cascaded layer, which fuses the output of the previous convolutional layer of the same dense block with the corresponding upsampled output of the lower dense block. Step S5.4: After processing the dense convolutional blocks in a series of skip paths, the final features of each block are fused with the features in the upsampled block to obtain the final prediction result. ,right Perform iterative denoising operations until a clear segmented image is obtained. until.

Images