Multilevel feature fusion medical image segmentation method and system based on diffusion model

Abstract

The invention discloses a multi-level feature fusion medical image segmentation method and system based on a diffusion model, and belongs to the technical field of medical image processing. The method comprises the following steps: acquiring an original medical skin lesion image and preprocessing the image, wherein the preprocessing comprises size adjustment and normalization; gaussian noise is added to the input image through the forward diffusion process, and the image with the noise added is obtained; constructing a denoising network encoder, and inputting the preprocessed original input image and the noise image into the denoising network encoder; and constructing a de-noising network decoder, sending the output features of the encoder into the decoder through jump connection and a multi-level feature fusion module, and outputting a segmentation result graph. According to the invention, through a boundary perception loss function and a multi-level feature fusion mechanism, the segmentation precision is improved, and the robustness of the model to noise is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a multi-level feature fusion medical image segmentation method and system based on a diffusion model. Background Art

[0002] The goal of skin lesion segmentation is to identify skin abnormalities or lesions from dermatological images. Skin lesions, including melanoma, basal cell carcinoma, squamous cell carcinoma, and other benign or malignant growths, have traditionally been diagnosed by visual inspection and manual delineation. Accurate and rapid segmentation of these lesions plays a vital role in early detection, treatment planning, and disease progression monitoring. Automated medical image segmentation methods have received significant attention in recent years for their ability to improve the accuracy and reliability of diagnostic results. However, for traditional segmentation network models, the denoising ability is limited, resulting in impaired denoising performance and segmentation accuracy in noisy environments; and for traditional segmentation network segmentation methods, all regions are treated equally without considering the importance of different regions, which often leads to inaccurate segmentation of boundary regions; finally, traditional segmentation networks use skip connections to enrich semantic information, but this semantic information is limited to the semantic information of feature maps at adjacent levels, without comprehensively considering the global multi-level feature map semantic information. Summary of the invention

[0003] The purpose of the present invention is to solve the problems existing in the existing medical image segmentation methods, such as weak denoising ability, sensitivity to lesion boundaries, insufficient semantic information, etc., and to provide a multi-level feature fusion medical image segmentation method and system based on a diffusion model.

[0004] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:

[0005] In a first aspect, the present invention provides a multi-level feature fusion medical image segmentation method based on a diffusion model, comprising the following steps:

[0006] S1: Acquire a medical image with skin lesions, and perform corresponding preprocessing on the medical image to obtain a preprocessed medical image;

[0007] S2: performing a forward diffusion process on the preprocessed medical image, and gradually adding Gaussian noise to obtain a medical image after adding noise;

[0008] S3: The medical image after adding noise and the medical image after preprocessing are input into the trained denoising network to obtain the segmentation result image of the skin lesion; the denoising network comprises a denoising network encoder, a multi-level feature fusion module, a bottleneck structure and a denoising network decoder, the denoising network encoder comprises five encoders whose parameters are not shared, each encoder comprises four residual blocks and two linear attention modules, the multi-level feature fusion module comprises ten fusion modules whose parameters are not shared, each fusion module comprises a spatial attention module and a channel attention module, the bottleneck structure comprises two residual blocks, a linear attention module and a self-attention module, and the denoising network decoder comprises five decoders whose parameters are not shared, each decoder comprises two residual blocks and a linear attention module;

[0009] In a denoising network encoder, the medical image after adding noise and the preprocessed medical image are used as inputs, and are processed by a first encoder to obtain a first feature map, a second feature map, a first noise feature map, and a first medical image feature map; the first noise feature map and the first medical image feature map are input into a second encoder for processing to obtain a third feature map, a fourth feature map, a second noise feature map, and a second medical image feature map; the second noise feature map and the second medical image feature map are input into a third encoder for processing to obtain a fifth feature map, a sixth feature map, a third noise feature map, and a third medical image feature map; the third noise feature map and the third medical image feature map are input into a fourth encoder for processing to obtain a seventh feature map, an eighth feature map, a fourth noise feature map, and a fourth medical image feature map; the fourth noise feature map and the fourth medical image feature map are input into a fifth encoder for processing to obtain a ninth feature map, a tenth feature map, a fifth noise feature map, and a fifth medical image feature map;

[0010] In the multi-level feature fusion module, the first feature map, the third feature map, the fifth feature map, the seventh feature map and the ninth feature map are used as the first input, and the second feature map, the fourth feature map, the sixth feature map, the eighth feature map and the tenth feature map are used as the second input. After the first fusion module processes the first input, a first multi-level fusion feature map is obtained. After the second fusion module processes the second input, a second multi-level fusion feature map is obtained. After the third fusion module processes the first input, a third multi-level fusion feature map is obtained. After the fourth fusion module processes the second input, a fourth multi-level fusion feature map is obtained. After the fifth fusion module processes the first input, a fifth multi-level fusion feature map is obtained. After the sixth fusion module processes the second input, a sixth multi-level fusion feature map is obtained. After the seventh fusion module processes the first input, a seventh multi-level fusion feature map is obtained. After the eighth fusion module processes the second input, an eighth multi-level fusion feature map is obtained. After the ninth fusion module processes the first input, a ninth multi-level fusion feature map is obtained. After the tenth fusion module processes the second input, a tenth multi-level fusion feature map is obtained.

[0011] In the bottleneck structure, the fifth noise feature map and the fifth medical image feature map are used as inputs, and a sixth noise feature map is output after processing;

[0012] In the denoising network decoder, the fifth decoder takes the sixth noise feature map, the tenth multi-level fusion feature map and the ninth multi-level fusion feature map as input, and after processing, obtains the seventh noise feature map. The fourth decoder takes the seventh noise feature map, the eighth multi-level fusion feature map and the seventh multi-level fusion feature map as input, and after processing, obtains the eighth noise feature map. The third decoder takes the eighth noise feature map, the sixth multi-level fusion feature map and the fifth multi-level fusion feature map as input, and after processing, obtains the ninth noise feature map. The second decoder takes the ninth noise feature map, the fourth multi-level fusion feature map and the third multi-level fusion feature map as input, and after processing, obtains the tenth noise feature map. The first decoder takes the tenth noise feature map, the second multi-level fusion feature map and the first multi-level fusion feature map as input, and after processing, obtains the segmentation result image of the skin lesions.

[0013] Based on the above scheme, each step can be implemented in the following preferred specific manner.

[0014] As a preferred embodiment of the above-mentioned first aspect, in step S1, the preprocessing is specifically: randomly cropping medical images with skin lesions of different sizes to obtain cropped medical images of a size of 128×128×3, and then normalizing the cropped medical images and performing horizontal mirror flipping, vertical mirror flipping and intensity change to obtain the preprocessed medical image.

[0015] As a preferred embodiment of the above-mentioned first aspect, in step S3, the processing flow in the first encoder is as follows: inputting the medical image and time series after adding noise into the first residual block to obtain a first intermediate feature map, inputting the preprocessed medical image and time series into the second residual block to obtain a second intermediate feature map, splicing the first intermediate feature map and the second intermediate feature map to obtain the first feature map, inputting the first feature map and the time series into the third residual block to obtain a third intermediate feature map, splicing the third intermediate feature map and the first intermediate feature map to obtain a fourth intermediate feature map, inputting the fourth intermediate feature map and the time series into the fourth residual block to obtain a fifth intermediate feature map, passing the fifth intermediate feature map through the first linear attention module to obtain the first noise feature map, splicing the first noise feature map and the third intermediate feature map to obtain the second feature map, and inputting the second feature map into the second linear attention module to obtain the first medical image feature map.

[0016] As a preferred embodiment of the above-mentioned first aspect, in step S3, the processing flow in the bottleneck structure is as follows: adding the fifth noise feature map and the fifth medical image feature map to obtain an added feature map, inputting the added feature map and the time series into the fifth residual block to obtain a sixth intermediate feature map, processing the sixth intermediate feature map through the third linear attention module to obtain a seventh intermediate feature map, processing the sixth intermediate feature map through the self-attention module to obtain an eighth intermediate feature map, adding the seventh intermediate feature map and the eighth intermediate feature map and inputting the sixth residual block to obtain the sixth noise feature map.

[0017] As a preferred embodiment of the above-mentioned first aspect, in step S3, the processing flow in each fusion module is as follows: each feature map in the first input or the second input is processed by a fusion branch respectively, each fusion branch outputs a fusion feature map, the fusion feature maps output by each fusion branch are multiplied to obtain a global fusion feature map, and the global fusion feature map is processed by a convolution layer with an activation function to obtain a multi-level fusion feature map.

[0018] As a preferred embodiment of the above-mentioned first aspect, in step S3, in each fusion branch, the input feature map is passed through a first spatial attention module to obtain a first spatial weight map, the input feature map and the first spatial weight map are multiplied to obtain a first spatial weighted feature map, the first spatial weighted feature map is passed through a first channel attention module to obtain a first channel weight map, the first spatial weighted feature map and the first channel weight map are multiplied to obtain a first channel weighted feature map, the first channel weighted feature map is subjected to an adaptive average pooling operation to obtain a pooled weighted feature map, the pooled weighted feature map is subjected to a layer of convolutional layer processing to obtain an output fused feature map.

[0019] As a preferred embodiment of the above-mentioned first aspect, in step S3, the processing flow in the fifth decoder is as follows: the sixth noise feature map and the tenth multi-level fusion feature map are spliced ​​to obtain a ninth intermediate feature map, the ninth intermediate feature map and the time series are input into the seventh residual block to obtain a tenth intermediate feature map, the tenth intermediate feature map and the ninth multi-level fusion feature map are spliced ​​to obtain an eleventh intermediate feature map, the eleventh intermediate feature map and the time series are input into the eighth residual block to obtain the eleventh intermediate feature map, the eleventh intermediate feature map is processed by the fourth linear attention module to obtain a twelfth intermediate feature map, and the twelfth intermediate feature map is upsampled to obtain the seventh noise feature map.

[0020] As a preferred embodiment of the above-mentioned first aspect, in step S3, the processing flow in each residual block is as follows: the input time series is processed through a fully connected layer and a SiLU activation function to obtain a scale scaling vector and a shift offset vector, the scale scaling vector, the shift offset vector and the input feature map are scaled and offset element by element to obtain a feature map incorporating time embedding information, the feature map incorporating time embedding information is sequentially processed through a first convolutional layer, a SiLU activation function, a second convolutional layer and a SiLU activation function to obtain a thirteenth intermediate feature map, the input feature map is processed through a third convolutional layer to obtain a fourteenth intermediate feature map, the thirteenth intermediate feature map and the fourteenth intermediate feature map are added to obtain a feature map output by the residual block.

[0021] In a second aspect, the present invention provides a multi-level feature fusion medical image segmentation system based on a diffusion model, comprising:

[0022] A data preprocessing module, used for acquiring a medical image with skin lesions, and performing corresponding preprocessing on the medical image to obtain a preprocessed medical image;

[0023] A diffusion process processing module is used to perform a forward diffusion process on the preprocessed medical image, and gradually add Gaussian noise to obtain a medical image after adding noise;

[0024] A result acquisition module is used to input the medical image after adding noise and the medical image after preprocessing into a trained denoising network to obtain a segmentation result image of the skin lesion; the denoising network comprises a denoising network encoder, a multi-level feature fusion module, a bottleneck structure and a denoising network decoder, the denoising network encoder comprises five encoders whose parameters are not shared, each encoder comprises four residual blocks and two linear attention modules, the multi-level feature fusion module comprises ten fusion modules whose parameters are not shared, each fusion module comprises a spatial attention module and a channel attention module, the bottleneck structure comprises two residual blocks, a linear attention module and a self-attention module, and the denoising network decoder comprises five decoders whose parameters are not shared, each decoder comprises two residual blocks and a linear attention module;

[0025] In a denoising network encoder, the medical image after adding noise and the preprocessed medical image are used as inputs, and are processed by a first encoder to obtain a first feature map, a second feature map, a first noise feature map, and a first medical image feature map; the first noise feature map and the first medical image feature map are input into a second encoder for processing to obtain a third feature map, a fourth feature map, a second noise feature map, and a second medical image feature map; the second noise feature map and the second medical image feature map are input into a third encoder for processing to obtain a fifth feature map, a sixth feature map, a third noise feature map, and a third medical image feature map; the third noise feature map and the third medical image feature map are input into a fourth encoder for processing to obtain a seventh feature map, an eighth feature map, a fourth noise feature map, and a fourth medical image feature map; the fourth noise feature map and the fourth medical image feature map are input into a fifth encoder for processing to obtain a ninth feature map, a tenth feature map, a fifth noise feature map, and a fifth medical image feature map;

[0026] In the multi-level feature fusion module, the first feature map, the third feature map, the fifth feature map, the seventh feature map and the ninth feature map are used as the first input, and the second feature map, the fourth feature map, the sixth feature map, the eighth feature map and the tenth feature map are used as the second input. After the first fusion module processes the first input, a first multi-level fusion feature map is obtained. After the second fusion module processes the second input, a second multi-level fusion feature map is obtained. After the third fusion module processes the first input, a third multi-level fusion feature map is obtained. After the fourth fusion module processes the second input, a fourth multi-level fusion feature map is obtained. After the fifth fusion module processes the first input, a fifth multi-level fusion feature map is obtained. After the sixth fusion module processes the second input, a sixth multi-level fusion feature map is obtained. After the seventh fusion module processes the first input, a seventh multi-level fusion feature map is obtained. After the eighth fusion module processes the second input, an eighth multi-level fusion feature map is obtained. After the ninth fusion module processes the first input, a ninth multi-level fusion feature map is obtained. After the tenth fusion module processes the second input, a tenth multi-level fusion feature map is obtained.

[0027] In the bottleneck structure, the fifth noise feature map and the fifth medical image feature map are used as inputs, and a sixth noise feature map is output after processing;

[0028] In the denoising network decoder, the fifth decoder takes the sixth noise feature map, the tenth multi-level fusion feature map and the ninth multi-level fusion feature map as input, and after processing, obtains the seventh noise feature map. The fourth decoder takes the seventh noise feature map, the eighth multi-level fusion feature map and the seventh multi-level fusion feature map as input, and after processing, obtains the eighth noise feature map. The third decoder takes the eighth noise feature map, the sixth multi-level fusion feature map and the fifth multi-level fusion feature map as input, and after processing, obtains the ninth noise feature map. The second decoder takes the ninth noise feature map, the fourth multi-level fusion feature map and the third multi-level fusion feature map as input, and after processing, obtains the tenth noise feature map. The first decoder takes the tenth noise feature map, the second multi-level fusion feature map and the first multi-level fusion feature map as input, and after processing, obtains the segmentation result image of the skin lesions.

[0029] In a third aspect, the present invention provides a computer electronic device, comprising: an image acquisition module, an image segmentation module, and a result output module;

[0030] An image acquisition module, used to acquire the preprocessed medical image required for the denoising network input and the medical image after adding noise;

[0031] An image segmentation module, used to obtain a trained denoising network obtained by the multi-level feature fusion medical image segmentation method based on a diffusion model as described in any solution of the first aspect above;

[0032] The result output module is used to input the preprocessed medical image obtained by the image acquisition module and the medical image after adding noise into the denoising network obtained by the image segmentation module, and output the segmentation result image of the skin lesion.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] Because in the skin lesion segmentation task, the image will be affected by multiple factors such as noise, thereby interfering with the positioning and segmentation of the lesion area, the present invention innovatively combines the denoising network and the segmentation network, uses the characteristics of the diffusion model to reduce the impact of noise to a minimum, and provides a powerful denoising capability. When segmenting skin lesions, the accuracy of the edge area is crucial. This method improves the model's sensitivity to edge pixels through an edge-aware loss function. This sensitivity to edges ensures the accuracy of segmentation, especially at small edges, which can be captured and described more accurately. In the encoder-decoder network, this method uses a multi-level feature fusion mechanism at the jump connection, so that each decoder can obtain richer details, from shallow details to deep details, and each level contributes its unique semantic information, which increases the semantic richness of the segmentation results and helps the model better understand the complex skin lesion structure. Because the network structure used in this method integrates features at different levels and shows a high sensitivity in edge segmentation, this enables the model to have better generalization capabilities in various types of skin lesion images. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the process structure of the method of the present invention.

[0036] Figure 2 This is a diagram of the encoder structure of the method of the present invention.

[0037] Figure 3 The decoder structure diagram of the method of the present invention.

[0038] Figure 4 This is a structural diagram of the multi-level feature fusion module of the method of the present invention.

[0039] Figure 5 This is a structural diagram of the fusion module of the method of the present invention.

[0040] Figure 6 This is a bottleneck module structure diagram of the method of the present invention.

[0041] Figure 7 The figure is a comparison diagram of the segmentation effect between the method of the present invention and different methods. DETAILED DESCRIPTION

[0042] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0043] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0044] In a preferred implementation of the present invention, the above-mentioned multi-level feature fusion medical image segmentation method based on diffusion model includes the following steps S1 to S5. The specific implementation process is described in detail below.

[0045] S1: Acquire a medical image with skin lesions, and perform corresponding preprocessing on the medical image to obtain a preprocessed medical image.

[0046] It should be noted that in step S1 of the present invention, the specific steps of preprocessing the above-mentioned medical images include: randomly cropping medical images with skin lesions of different sizes to obtain cropped medical images of a size of 128×128×3, and before inputting the denoising network, normalizing the cropped medical images and performing data enhancement by methods such as horizontal mirror flipping, vertical mirror flipping, and intensity change to obtain preprocessed medical images.

[0047] The above normalization is calculated as follows:

[0048]

[0049] Among them, y k is the normalized value; x k is the value before normalization; X is the input sequence, that is, the cropped medical image; std(X) is the standard deviation of the input sequence, that is, the standard deviation of the cropped medical image; is the mean of the input sequence and the mean of the cropped medical image.

[0050] S2: Perform a forward diffusion process on the preprocessed medical image and gradually add Gaussian noise to obtain a medical image after adding noise.

[0051] It should be noted that in step S2, Gaussian noise is added to the preprocessed medical image by using the forward diffusion process to simulate the noise in the image acquisition process in the real world to obtain the medical image after adding noise. The specific operation of the forward diffusion process is as follows: the preprocessed medical image is continuously added with Gaussian noise to the current image, and the intensity of the Gaussian noise is adjusted by To adjust, defined as:

[0052]

[0053] Where x0 is the input preprocessed medical image; x t is an image with Gaussian noise at time t; q(x t |x0) represents the image sequence after adding Gaussian noise; N represents normal distribution; represents the noise variance scheduling; I is the unit matrix; ε represents the normally distributed random variable; β t represents the noise increment; α t represents the noise variance adjustment coefficient; α j represents the noise reduction coefficient of time step j; N(0,I) represents the normal distribution with mean 0 and variance as the unit matrix I; t represents the number of time steps.

[0054] S3: The medical image after adding noise and the medical image after preprocessing are input into the trained denoising network to obtain the segmentation result image of the skin lesion; the denoising network comprises a denoising network encoder, a multi-level feature fusion module, a bottleneck structure and a denoising network decoder, the denoising network encoder comprises five encoders whose parameters are not shared, each encoder comprises four residual blocks and two linear attention modules, the multi-level feature fusion module comprises ten fusion modules whose parameters are not shared, each fusion module comprises a spatial attention module and a channel attention module, the bottleneck structure comprises two residual blocks, a linear attention module and a self-attention module, and the denoising network decoder comprises five decoders whose parameters are not shared, each decoder comprises two residual blocks and a linear attention module;

[0055] In the denoising network encoder, the medical image after adding noise and the preprocessed medical image are taken as input, and after being processed by the first encoder, a first feature map h1, a second feature map b1, a first noise feature map x1′ and a first medical image feature map g1′ are obtained. The first noise feature map x1′ and the first medical image feature map g1′ are input into the second encoder for processing to obtain a third feature map h2, a fourth feature map b2, a second noise feature map x2′ and a second medical image feature map g2′. The second noise feature map x2′ and the second medical image feature map g2′ are input into the third encoder Processing to obtain the fifth characteristic map h3, the sixth characteristic map b3, the third noise characteristic map x3′ and the third medical image characteristic map g3′, the third noise characteristic map x3′ and the third medical image feature map g3′ are input to the fourth encoder for processing to obtain the seventh characteristic map h4, the eighth characteristic map b4, the fourth noise characteristic map x4′ and the fourth medical image feature map g4′, the fourth noise characteristic map x4′ and the fourth medical image feature map g4′ are input to the fifth encoder for processing to obtain the ninth characteristic map h5, the tenth characteristic map b5, the fifth noise characteristic map x5′ and the fifth medical image feature map g5′.

[0056] In the multi-level feature fusion module, the first feature map h1, the third feature map h2, the fifth feature map h3, the seventh feature map h4 and the ninth feature map h5 are taken as the first input, and the second feature map b1, the fourth feature map b2, the sixth feature map b3, the eighth feature map b4 and the tenth feature map b5 are taken as the second input. After the first fusion module processes the first input, the first multi-level fusion feature map h1′ is obtained. After the second fusion module processes the second input, the second multi-level fusion feature map b1′ is obtained. After the third fusion module processes the first input, the third multi-level fusion feature map h2′ is obtained. After the fourth fusion module processes the first input, the fourth multi-level fusion feature map h3′ is obtained. After processing the second input, the fourth multi-level fusion feature map b2′ is obtained. After the fifth fusion module processes the first input, the fifth multi-level fusion feature map h3′ is obtained. After the sixth fusion module processes the second input, the sixth multi-level fusion feature map b3′ is obtained. After the seventh fusion module processes the first input, the seventh multi-level fusion feature map h4′ is obtained. After the eighth fusion module processes the second input, the eighth multi-level fusion feature map b4′ is obtained. After the ninth fusion module processes the first input, the ninth multi-level fusion feature map h5′ is obtained. After the tenth fusion module processes the second input, the tenth multi-level fusion feature map b5′ is obtained.

[0057] In the bottleneck structure, the fifth noise feature map x5′ and the fifth medical image feature map g5′ are taken as input, and the sixth noise feature map x6′ is output after processing.

[0058] In the denoising network decoder, the fifth decoder takes the sixth noise feature map x6′, the tenth multi-level fusion feature map b5′ and the ninth multi-level fusion feature map h5′ as input, and after processing, obtains the seventh noise feature map x7′, the fourth decoder takes the seventh noise feature map x7′, the eighth multi-level fusion feature map b4′ and the seventh multi-level fusion feature map h4′ as input, and after processing, obtains the eighth noise feature map x8′, the third decoder takes the eighth noise feature map x8′, the sixth multi-level fusion feature map b3′ and the fifth multi-level fusion feature map h3′ as input, and after processing, obtains the ninth noise feature map x9′, the second decoder takes the ninth noise feature map x9′, the fourth multi-level fusion feature map b2′ and the third multi-level fusion feature map h2′ as input, and after processing, obtains the tenth noise feature map x10′, the first decoder takes the tenth noise feature map x10′, the second multi-level fusion feature map b1′ and the first multi-level fusion feature map h1′ as input, and after processing, obtains the segmentation result image of the skin lesions.

[0059] It should be noted that the present invention constructs a denoising network encoder, and inputs the preprocessed medical image and the medical image after adding noise into the denoising network encoder to obtain feature map information at different levels. The denoising network encoder includes five encoders whose parameters are not shared, and its structure is similar to the U-Net encoder part. Each encoder in the denoising network encoder includes two independent feature extraction paths, one for extracting features from the preprocessed medical image and the other for extracting features from the medical image after adding noise. Each path uses a deep convolutional network to learn the complex relationship between image content and noise patterns.

[0060] The denoising process in the denoising network is as follows:

[0061]

[0062] in, is the normal skin lesion image output by the denoising network, γ t represents the noise variance adjustment coefficient; ∈ θ represents the trained denoising network.

[0063] In this embodiment, the input of the denoising network encoder is a preprocessed medical image and a medical image after adding noise, and a dual path is used to extract features from the two images. The following takes the first encoder as an example to specifically describe the processing flow therein. The processing flows in the second encoder, the third encoder, the fourth encoder, and the fifth encoder are all consistent with the first encoder. The only difference is that the medical image after adding noise is replaced with a noise feature map of each scale, and the preprocessed medical image is replaced with a medical image feature map of each scale. The second noise feature map, the third noise feature map, the fourth noise feature map, and the fifth noise feature map can be generated in the same manner as the first noise feature map, and the second medical image feature map, the third medical image feature map, the fourth medical image feature map, and the fifth medical image feature map can be generated in the same manner as the first medical image feature map. Figure 2 As shown, in the first encoder, the medical image and time series after adding noise are first input into the first residual block to obtain the first intermediate feature map, the preprocessed medical image and time series are input into the second residual block to obtain the second intermediate feature map, the first intermediate feature map and the second intermediate feature map are spliced ​​to obtain the first feature map, the first feature map and the time series are input into the third residual block to obtain the third intermediate feature map, the third intermediate feature map and the first intermediate feature map are spliced ​​to obtain the fourth intermediate feature map, the fourth intermediate feature map and the time series are input into the fourth residual block to obtain the fifth intermediate feature map, the fifth intermediate feature map is passed through the first linear attention module to obtain the first noise feature map, the first noise feature map and the third intermediate feature map are spliced ​​to obtain the second feature map, and the second feature map is input into the second linear attention module to obtain the first medical image feature map.

[0064] It should be noted that the implementation method of the linear attention module belongs to the prior art and will not be described in detail in the present invention.

[0065] It should be noted that the present invention constructs a bottleneck structure, wherein the processing flow is as follows: Figure 6 As shown, the fifth noise feature map and the fifth medical image feature map are added to obtain an added feature map, the added feature map and the time series are input into the fifth residual block to obtain a sixth intermediate feature map, the sixth intermediate feature map is processed by the third linear attention module to obtain a seventh intermediate feature map, the sixth intermediate feature map is processed by the self-attention module to obtain an eighth intermediate feature map, the seventh intermediate feature map and the eighth intermediate feature map are added and input into the sixth residual block to obtain the sixth noise feature map.

[0066] It should be noted that the implementation method of the self-attention module belongs to the prior art and will not be described in detail in the present invention.

[0067] It should be noted that the present invention constructs a multi-level feature fusion module, which fuses all the feature maps output by the denoising network encoder to enrich the semantic information at each level. The multi-level feature fusion module includes ten fusion modules whose parameters are not shared, and each fusion module includes a spatial attention module and a channel attention module. The channel attention module (ChannelAttention) further strengthens the response to important channels and weakens the influence of unimportant channels. This module uses adaptive average pooling and maximum pooling to extract global features and converts them through fully connected layers. The spatial attention module (SpatialAttention) emphasizes the features of important spatial areas and improves the segmentation performance of the model by focusing on specific spatial positions. For each level of jump connection between the encoder and the decoder, an independent fusion module is configured. When this module is initialized, a series of convolutional layers will be generated to process the feature maps of each level. It is used to receive the output feature map of the encoder of the corresponding level. Through adaptive average pooling or upsampling, the resolution of the feature map is unified to the same size as the output feature map of the encoder of the current level. Figure 4 As shown, the first fusion module ( Figure 4 The fusion module H1 in the third fusion module ( Figure 4 The fusion module H2 in the fifth fusion module ( Figure 4 The fusion module H3 in the seventh fusion module ( Figure 4 The fusion module H4 in the Figure 4 The fusion module H5 in the figure is used to process the first input (the first feature map h1, the third feature map h2, the fifth feature map h3, the seventh feature map h4 and the ninth feature map h5), and each fusion module obtains the first multi-level fusion feature map h1′, the third multi-level fusion feature map h2′, the fifth multi-level fusion feature map h3′, the seventh multi-level fusion feature map h4′ and the ninth multi-level fusion feature map h5′; the second fusion module ( Figure 4 The fusion module B1 in the fourth fusion module ( Figure 4 Fusion module B2 in the sixth fusion module ( Figure 4 The fusion module B3 in the eighth fusion module ( Figure 4 The fusion module B4 in the Figure 4 The fusion module B5) is used to process the second input (the second feature map b1, the fourth feature map b2, the sixth feature map b3, the eighth feature map b4 and the tenth feature map b5), and each fusion module correspondingly obtains the second multi-level fusion feature map b1′, the fourth multi-level fusion feature map b2′, the sixth multi-level fusion feature map b3′, the eighth multi-level fusion feature map b4′ and the tenth multi-level fusion feature map b5′.

[0068] In this embodiment, the processing flow in each fusion module is as follows: Figure 5 As shown, each feature map in the first input or the second input is processed by a fusion branch, each fusion branch outputs a fusion feature map, and the fusion feature maps output by each fusion branch are multiplied to obtain a feature map that integrates multi-level information, that is, a global fusion feature map. The global fusion feature map is processed by a 1×1 convolution layer ReLU (Conv(·)) with an activation function to convert the global fusion feature map Converted into an output feature map with a predetermined number of channels, i.e., a multi-level fusion feature map F out :

[0069]

[0070] Among them, the 1×1 convolution layer is used to convert the global fusion feature map into a predetermined number of output channels, and then a nonlinear transformation is performed through the ReLU activation function to provide an enhanced feature map for the decoder.

[0071] In each fusion branch, for the encoder at the lth level, set its output feature map to F l , the input feature map F l Through the spatial attention module SA(·), we get the spatial weight map SA(F l ), and then the input feature map F l and the spatial weight map SA(F l ) to obtain the spatial weighted feature map

[0072]

[0073] The spatial weighted feature map is passed through the channel attention module CA(·) to obtain the channel weight map The spatially weighted feature map Multiply it with the channel weight map to get the channel weighted feature map:

[0074]

[0075] The channel weighted feature map is subjected to an adaptive average pooling operation to obtain a pooled weighted feature map, and the pooled weighted feature map is processed by a 3×3 convolution layer to obtain an output fused feature map.

[0076] It should be noted that the present invention constructs a denoising network decoder, and the two feature maps (h and b) output by each level encoder are passed to the decoder at the same level through a jump connection, and the feature map x and feature map g output by the encoder are passed to the encoder at the next level. In the denoising network decoder, the sixth noise feature map is used as input and the segmentation result image is output. The denoising network decoder is similar to the U-Net network decoding structure, and includes five decoders whose parameters are not shared. Each decoder performs an upsampling operation to obtain a sampled feature map, and then the decoder splices the multi-level fusion feature map output by the multi-level feature fusion module through a jump connection, and continues the upsampling operation to reconstruct the resolution size of the original image. The fifth decoder is taken as an example to describe the processing flow in detail. The processing flows in the fourth decoder, the third decoder, the second decoder and the first decoder are consistent with those in the fifth encoder. The only difference is that the input sixth noise feature map is replaced with the seventh noise feature map, the eighth noise feature map, the ninth noise feature map and the tenth noise feature map, which are used as part of the input to the fourth decoder, the third decoder, the second decoder and the first decoder respectively, and the tenth multi-level fusion feature map and the ninth multi-level fusion feature map are replaced with the multi-level fusion feature maps output by the other two fusion modules, which are used as another part of the input to the fourth decoder, the third decoder, the second decoder and the first decoder respectively. In this way, the eighth noise feature map, the ninth noise feature map, the tenth noise feature map and the segmentation result image of the skin lesion can be generated in the same way as the seventh noise feature map is output by the fifth decoder. Figure 3 As shown, in the fifth decoder, the sixth noise feature map and the tenth multi-level fusion feature map are spliced ​​to obtain a ninth intermediate feature map, the ninth intermediate feature map and the time series are input into the seventh residual block to obtain a tenth intermediate feature map, the tenth intermediate feature map and the ninth multi-level fusion feature map are spliced ​​to obtain an eleventh intermediate feature map, the eleventh intermediate feature map and the time series are input into the eighth residual block to obtain the eleventh intermediate feature map, the eleventh intermediate feature map is processed by the fourth linear attention module to obtain the twelfth intermediate feature map, the twelfth intermediate feature map is upsampled to obtain the seventh noise feature map.

[0077] It should be noted that in step S3, the processing flow in each residual block is as follows: the input time series is processed through a fully connected layer and a SiLU activation function to obtain a scale vector and a shift vector, the scale vector, the shift vector and the input feature map are scaled and offset element by element to obtain a feature map that incorporates time embedding information, the feature map that incorporates time embedding information is processed through a first convolutional layer, a SiLU activation function, a second convolutional layer and a SiLU activation function to obtain a thirteenth intermediate feature map, the input feature map is processed through a third convolutional layer to obtain a fourteenth intermediate feature map, the thirteenth intermediate feature map and the fourteenth intermediate feature map are added to obtain a feature map output by the residual block.

[0078] It should be noted that in step S3, when training the denoising network, the parameters in the denoising network encoder and the denoising network decoder are optimized using the Adam optimizer. The present invention uses two types of loss functions to optimize the denoising network, one type of loss function is used to optimize the segmentation performance of the denoising network, and the other type of loss function is used to optimize the denoising performance of the denoising network. The total loss function of the present invention comprehensively uses the pixel-level loss function and the boundary-aware loss function, compares the predicted mask with the true label, and accurately extracts the lesion area from the image.

[0079] In this embodiment, the pixel-level loss function L seg By the cross entropy loss function L ce and Dice loss function L Dice The cross entropy loss function L is composed by weighted combination. ce , used to optimize pixel-level classification accuracy: Dice loss function L Dice , used for the denoising network to identify the overall shape and size of the segmented area and improve the consistency of segmentation. The details are as follows:

[0080] L seg =aL ce +bL Dice

[0081]

[0082] In the formula, a and b are hyperparameters; y i represents the true label of the i-th pixel, represents the probability predicted by the denoising network, and N is the total number of pixels in the image.

[0083] In this embodiment, the boundary perception loss function L noise The performance specifically in identifying the edges of skin lesions, by emphasizing the prediction accuracy of the boundary area to enhance the detail performance of the segmentation, can be expressed as:

[0084] L noise =(1+αW θ )‖e-∈(x t ,g,t)‖ 2

[0085] Where e is the noise predicted by the denoising network, ∈(x t ,g,t) represents the given image x at time t t and the true noise and target prediction value under condition g; W θ represents a dynamic parameter, which aims to increase the weight of noise prediction in the boundary area and reduce the weight as it moves from the boundary to the inside; α is a weight parameter used to adjust the dynamic parameter W θ impact.

[0086] In summary, the total loss function is:

[0087] L=L seg +L noise

[0088] The present invention will now use a specific example to demonstrate the application effect of the multi-level feature fusion medical image segmentation method based on the diffusion model described in the above embodiment on a specific data set, so as to facilitate understanding of the essence of the present invention.

[0089] Example

[0090] To verify the effectiveness of the present invention, this embodiment evaluates the performance of the proposed method on the ISIC2018 dataset. The ISIC 2018 dataset comes from the International Skin Imaging Collaboration and is a comprehensive dataset designed for tasks such as instance segmentation, semantic segmentation, and object detection in medical research. The dataset contains a large number of dermoscopic images, which are essential for studying skin lesions. The categories included in the dataset are skin cancer, pigmented nets, globules, miliary cysts, negative networks, and stripes. Each image has pixel-level instance segmentation annotations, and the present invention randomly divides the test set in a ratio of 7:2:1.

[0091] The method of the present invention is developed under the Pytorch framework and trained on three 3090 graphics cards. During the training, the Adam optimizer is used to optimize the parameters of the denoising network. During the training, the batch size is set to 24, the initial learning rate is set to 0.00005, the learning rate decay algorithm is adopted, the patience value is 9, the decay factor is 0.5, and a total of 400 rounds of training are performed.

[0092] The present invention uses Dice, Sensitivity, Specificity, and Accuracy to evaluate the segmentation performance of the denoising network. Dice measures the segmentation performance by calculating the overlap rate between the predicted result and the true label boundary. The larger the Dice coefficient value, the closer the segmentation result is to the true label. Sensitivity is also called recall rate, which measures the ratio of the positive examples (such as lesion areas) correctly identified by the denoising network to the total number of actual positive examples. A high sensitivity score indicates that the denoising network has captured all the real positive examples well. Specificity measures the ratio of the negative examples (such as healthy tissues) correctly identified by the denoising network to the total number of actual negative examples. A high specificity score indicates that the denoising network has well avoided incorrectly marking negative examples as positive examples. Accuracy is the ratio of the positive and negative examples correctly predicted by the denoising network to the total number of all predictions. Accuracy gives an indicator of the overall performance of the model.

[0093] On the ISIC2018 dataset, the method of the present invention is compared with medical image segmentation methods such as U-Net, DAGAN, TransUnet, Swin-Unet, DeepLabv3+, Att-UNet, UCTransUNet, MissFormer, EnsDiff, and the experimental results are shown in Table 1. It can be seen that the Dice, Sensitivity, Specificity, and Accuracy of the present invention on the ISIC2018 dataset are 0.9105, 0.9104, 0.9752, and 0.9612, respectively, which proves the effectiveness of the method of the present invention in segmenting skin lesion areas and boundaries.

[0094] Table 1. Objective evaluation comparison of different methods on the ISIC2018 dataset

[0095] method DSC SE SP ACC U-Net 0.8545 0.8800 0.9697 0.9404 DAGAN 0.8807 0.9072 0.9588 0.9324 TransUnet 0.8499 0.8578 0.9653 0.9452 Swin-Unet 0.8946 0.9056 0.9798 0.9645 DeepLabv3+ 0.8820 0.8560 0.9770 0.9510 Att-UNet 0.8566 0.8674 0.9863 0.9376 UCTransUNet 0.8838 0.9825 0.8429 0.9527 MissFormer 0.8631 0.9690 0.8458 0.9427 EnsDiff 0.8775 0.8358 0.9812 0.9502 Method of the present invention 0.9105 0.9104 0.9752 0.9612

[0096] Furthermore, the present invention compares the segmentation results of TransUNet, EnsDiff, and U-Net visually. Figure 7 As shown, it can be seen that the method proposed in the present invention can more accurately identify and locate the skin lesion area and boundary, while the segmentation results of other methods are rougher.

[0097] It should be noted that the above-mentioned multi-level feature fusion medical image segmentation method based on the diffusion model of the present invention can be used for non-disease diagnosis purposes, such as scientific research, detection or data processing, and can also be used to provide intermediate parameters in the field of auxiliary diagnosis.

[0098] It should also be noted that the multi-level feature fusion medical image segmentation method based on the diffusion model in the above embodiment can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a multi-level feature fusion medical image segmentation system based on a diffusion model corresponding to the multi-level feature fusion medical image segmentation method based on the diffusion model provided in the above embodiment, including:

[0099] A data preprocessing module, used for acquiring a medical image with skin lesions, and performing corresponding preprocessing on the medical image to obtain a preprocessed medical image;

[0100] A diffusion process processing module is used to perform a forward diffusion process on the preprocessed medical image, and gradually add Gaussian noise to obtain a medical image after adding noise;

[0101] A result acquisition module is used to input the medical image after adding noise and the medical image after preprocessing into a trained denoising network to obtain a segmentation result image of the skin lesion; the denoising network comprises a denoising network encoder, a multi-level feature fusion module, a bottleneck structure and a denoising network decoder, the denoising network encoder comprises five encoders whose parameters are not shared, each encoder comprises four residual blocks and two linear attention modules, the multi-level feature fusion module comprises ten fusion modules whose parameters are not shared, each fusion module comprises a spatial attention module and a channel attention module, the bottleneck structure comprises two residual blocks, a linear attention module and a self-attention module, and the denoising network decoder comprises five decoders whose parameters are not shared, each decoder comprises two residual blocks and a linear attention module;

[0102] In a denoising network encoder, the medical image after adding noise and the preprocessed medical image are used as inputs, and are processed by a first encoder to obtain a first feature map, a second feature map, a first noise feature map, and a first medical image feature map; the first noise feature map and the first medical image feature map are input into a second encoder for processing to obtain a third feature map, a fourth feature map, a second noise feature map, and a second medical image feature map; the second noise feature map and the second medical image feature map are input into a third encoder for processing to obtain a fifth feature map, a sixth feature map, a third noise feature map, and a third medical image feature map; the third noise feature map and the third medical image feature map are input into a fourth encoder for processing to obtain a seventh feature map, an eighth feature map, a fourth noise feature map, and a fourth medical image feature map; the fourth noise feature map and the fourth medical image feature map are input into a fifth encoder for processing to obtain a ninth feature map, a tenth feature map, a fifth noise feature map, and a fifth medical image feature map;

[0103] In the multi-level feature fusion module, the first feature map, the third feature map, the fifth feature map, the seventh feature map and the ninth feature map are used as the first input, and the second feature map, the fourth feature map, the sixth feature map, the eighth feature map and the tenth feature map are used as the second input. After the first fusion module processes the first input, a first multi-level fusion feature map is obtained. After the second fusion module processes the second input, a second multi-level fusion feature map is obtained. After the third fusion module processes the first input, a third multi-level fusion feature map is obtained. After the fourth fusion module processes the second input, a fourth multi-level fusion feature map is obtained. After the fifth fusion module processes the first input, a fifth multi-level fusion feature map is obtained. After the sixth fusion module processes the second input, a sixth multi-level fusion feature map is obtained. After the seventh fusion module processes the first input, a seventh multi-level fusion feature map is obtained. After the eighth fusion module processes the second input, an eighth multi-level fusion feature map is obtained. After the ninth fusion module processes the first input, a ninth multi-level fusion feature map is obtained. After the tenth fusion module processes the second input, a tenth multi-level fusion feature map is obtained.

[0104] In the bottleneck structure, the fifth noise feature map and the fifth medical image feature map are used as inputs, and a sixth noise feature map is output after processing;

[0105] In the denoising network decoder, the fifth decoder takes the sixth noise feature map, the tenth multi-level fusion feature map and the ninth multi-level fusion feature map as input, and after processing, obtains the seventh noise feature map. The fourth decoder takes the seventh noise feature map, the eighth multi-level fusion feature map and the seventh multi-level fusion feature map as input, and after processing, obtains the eighth noise feature map. The third decoder takes the eighth noise feature map, the sixth multi-level fusion feature map and the fifth multi-level fusion feature map as input, and after processing, obtains the ninth noise feature map. The second decoder takes the ninth noise feature map, the fourth multi-level fusion feature map and the third multi-level fusion feature map as input, and after processing, obtains the tenth noise feature map. The first decoder takes the tenth noise feature map, the second multi-level fusion feature map and the first multi-level fusion feature map as input, and after processing, obtains the segmentation result image of the skin lesions.

[0106] Based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the multi-level feature fusion medical image segmentation method based on the diffusion model provided in the above embodiment, including: an image acquisition module, an image segmentation module and a result output module;

[0107] An image acquisition module, used to acquire the preprocessed medical image required for the denoising network input and the medical image after adding noise;

[0108] An image segmentation module, used to obtain a trained denoising network obtained by the multi-level feature fusion medical image segmentation method based on the diffusion model as described in the above embodiment;

[0109] The result output module is used to input the preprocessed medical image obtained by the image acquisition module and the medical image after adding noise into the denoising network obtained by the image segmentation module, and output the segmentation result image of the skin lesion.

[0110] It should be noted that the image acquisition module acquires the preprocessed medical image and the medical image after adding noise, and the image segmentation module acquires a trained denoising network, and both can be acquired online or offline. For online acquisition, the image acquisition module can capture the preprocessed medical image and add noise to it by online controlling the imaging device to generate the medical image after adding noise, and the image segmentation module can train the denoising network by online training. For offline acquisition, the image acquisition module can acquire the preprocessed medical image and the medical image after adding noise that have been acquired and stored by the imaging device, and the image segmentation module can read the denoising network stored on the storage medium after training. The result output module is used to output the segmentation result image of the skin lesion. In actual implementation, the result can be output by the local display, or uploaded to the cloud for output for remote display or reading.

[0111] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A multi-level feature fusion medical image segmentation method based on diffusion model, characterized in that: The following steps are involved: S1: Acquire a medical image with skin lesions, and perform corresponding preprocessing on the medical image to obtain a preprocessed medical image; S2: performing a forward diffusion process on the preprocessed medical image, and gradually adding Gaussian noise to obtain a medical image after adding noise; S3: The medical image after adding noise and the medical image after preprocessing are input into the trained denoising network to obtain the segmentation result image of the skin lesion; The denoising network includes a denoising network encoder, a multi-level feature fusion module, a bottleneck structure and a denoising network decoder. The denoising network encoder includes five encoders whose parameters are not shared, each encoder includes four residual blocks and two linear attention modules, the multi-level feature fusion module includes ten fusion modules whose parameters are not shared, each fusion module includes a spatial attention module and a channel attention module, the bottleneck structure includes two residual blocks, a linear attention module and a self-attention module, and the denoising network decoder includes five decoders whose parameters are not shared, each decoder includes two residual blocks and a linear attention module; In a denoising network encoder, the medical image after adding noise and the preprocessed medical image are used as inputs, and are processed by a first encoder to obtain a first feature map, a second feature map, a first noise feature map, and a first medical image feature map; the first noise feature map and the first medical image feature map are input into a second encoder for processing to obtain a third feature map, a fourth feature map, a second noise feature map, and a second medical image feature map; the second noise feature map and the second medical image feature map are input into a third encoder for processing to obtain a fifth feature map, a sixth feature map, a third noise feature map, and a third medical image feature map; the third noise feature map and the third medical image feature map are input into a fourth encoder for processing to obtain a seventh feature map, an eighth feature map, a fourth noise feature map, and a fourth medical image feature map; the fourth noise feature map and the fourth medical image feature map are input into a fifth encoder for processing to obtain a ninth feature map, a tenth feature map, a fifth noise feature map, and a fifth medical image feature map; In the multi-level feature fusion module, the first feature map, the third feature map, the fifth feature map, the seventh feature map and the ninth feature map are used as the first input, and the second feature map, the fourth feature map, the sixth feature map, the eighth feature map and the tenth feature map are used as the second input. After the first fusion module processes the first input, a first multi-level fusion feature map is obtained. After the second fusion module processes the second input, a second multi-level fusion feature map is obtained. After the third fusion module processes the first input, a third multi-level fusion feature map is obtained. After the fourth fusion module processes the second input, a fourth multi-level fusion feature map is obtained. After the fifth fusion module processes the first input, a fifth multi-level fusion feature map is obtained. After the sixth fusion module processes the second input, a sixth multi-level fusion feature map is obtained. After the seventh fusion module processes the first input, a seventh multi-level fusion feature map is obtained. After the eighth fusion module processes the second input, an eighth multi-level fusion feature map is obtained. After the ninth fusion module processes the first input, a ninth multi-level fusion feature map is obtained. After the tenth fusion module processes the second input, a tenth multi-level fusion feature map is obtained. In the bottleneck structure, the fifth noise feature map and the fifth medical image feature map are used as inputs, and a sixth noise feature map is output after processing; In the denoising network decoder, the fifth decoder takes the sixth noise feature map, the tenth multi-level fusion feature map and the ninth multi-level fusion feature map as input, and after processing, obtains the seventh noise feature map. The fourth decoder takes the seventh noise feature map, the eighth multi-level fusion feature map and the seventh multi-level fusion feature map as input, and after processing, obtains the eighth noise feature map. The third decoder takes the eighth noise feature map, the sixth multi-level fusion feature map and the fifth multi-level fusion feature map as input, and after processing, obtains the ninth noise feature map. The second decoder takes the ninth noise feature map, the fourth multi-level fusion feature map and the third multi-level fusion feature map as input, and after processing, obtains the tenth noise feature map. The first decoder takes the tenth noise feature map, the second multi-level fusion feature map and the first multi-level fusion feature map as input, and after processing, obtains the segmentation result image of the skin lesions.

2. The multi-level feature fusion medical image segmentation method based on diffusion model as claimed in claim 1, characterized in that: In step S1, the preprocessing is specifically as follows: randomly cropping medical images with skin lesions of different sizes to obtain cropped medical images of a size of 128×128×3, and then normalizing the cropped medical images and performing horizontal mirror flipping, vertical mirror flipping, and intensity change to obtain preprocessed medical images.

3. The multi-level feature fusion medical image segmentation method based on diffusion model as claimed in claim 1, characterized in that: In step S3, the processing flow in the first encoder is as follows: the medical image and time series after adding noise are input into the first residual block to obtain a first intermediate feature map, the preprocessed medical image and time series are input into the second residual block to obtain a second intermediate feature map, the first intermediate feature map and the second intermediate feature map are spliced ​​to obtain the first feature map, the first feature map and the time series are input into the third residual block to obtain a third intermediate feature map, the third intermediate feature map and the first intermediate feature map are spliced ​​to obtain a fourth intermediate feature map, the fourth intermediate feature map and the time series are input into the fourth residual block to obtain a fifth intermediate feature map, the fifth intermediate feature map is passed through the first linear attention module to obtain the first noise feature map, the first noise feature map and the third intermediate feature map are spliced ​​to obtain the second feature map, and the second feature map is input into the second linear attention module to obtain the first medical image feature map.

4. The multi-level feature fusion medical image segmentation method based on diffusion model as claimed in claim 1, characterized in that: In step S3, the processing flow in the bottleneck structure is as follows: adding the fifth noise feature map and the fifth medical image feature map to obtain the added feature map, inputting the added feature map and the time series into the fifth residual block to obtain the sixth intermediate feature map, processing the sixth intermediate feature map through the third linear attention module to obtain the seventh intermediate feature map, processing the sixth intermediate feature map through the self-attention module to obtain the eighth intermediate feature map, adding the seventh intermediate feature map and the eighth intermediate feature map and inputting the sixth residual block to obtain the sixth noise feature map.

5. The multi-level feature fusion medical image segmentation method based on diffusion model as claimed in claim 1, characterized in that: In step S3, the processing flow in each fusion module is as follows: each feature map in the first input or the second input is processed by a fusion branch respectively, each fusion branch outputs a fusion feature map, the fusion feature maps output by each fusion branch are multiplied to obtain a global fusion feature map, and the global fusion feature map is processed by a convolution layer with an activation function to obtain a multi-level fusion feature map.

6. The multi-level feature fusion medical image segmentation method based on diffusion model as claimed in claim 5, characterized in that: In step S3, in each fusion branch, the input feature map is passed through the first spatial attention module to obtain the first spatial weight map, the input feature map and the first spatial weight map are multiplied to obtain the first spatial weighted feature map, the first spatial weighted feature map is passed through the first channel attention module to obtain the first channel weight map, the first spatial weighted feature map and the first channel weight map are multiplied to obtain the first channel weighted feature map, the first channel weighted feature map is subjected to an adaptive average pooling operation to obtain a pooled weighted feature map, the pooled weighted feature map is subjected to a convolutional layer to obtain an output fusion feature map.

7. The multi-level feature fusion medical image segmentation method based on diffusion model as claimed in claim 1, characterized in that: In step S3, the processing flow in the fifth decoder is as follows: the sixth noise feature map and the tenth multi-level fusion feature map are spliced ​​to obtain a ninth intermediate feature map, the ninth intermediate feature map and the time series are input into the seventh residual block to obtain a tenth intermediate feature map, the tenth intermediate feature map and the ninth multi-level fusion feature map are spliced ​​to obtain an eleventh intermediate feature map, the eleventh intermediate feature map and the time series are input into the eighth residual block to obtain the eleventh intermediate feature map, the eleventh intermediate feature map is processed by the fourth linear attention module to obtain a twelfth intermediate feature map, and the twelfth intermediate feature map is upsampled to obtain a seventh noise feature map.

8. The multi-level feature fusion medical image segmentation method based on diffusion model as claimed in claim 1, characterized in that: In step S3, the processing flow in each residual block is as follows: the input time series is processed through a fully connected layer and a SiLU activation function to obtain a scale vector and a shift vector, the scale vector, the shift vector and the input feature map are scaled and shifted element by element to obtain a feature map incorporating time embedding information, the feature map incorporating time embedding information is sequentially processed through the first convolution layer, the SiLU activation function, the second convolution layer and the SiLU activation function to obtain a thirteenth intermediate feature map, the input feature map is processed through the third convolution layer to obtain a fourteenth intermediate feature map, the thirteenth intermediate feature map and the fourteenth intermediate feature map are added to obtain the feature map output by the residual block.

9. A multi-level feature fusion medical image segmentation system based on diffusion model, characterized in that: include: A data preprocessing module, used for acquiring a medical image with skin lesions, and performing corresponding preprocessing on the medical image to obtain a preprocessed medical image; A diffusion process processing module is used to perform a forward diffusion process on the preprocessed medical image, and gradually add Gaussian noise to obtain a medical image after adding noise; A result acquisition module is used to input the medical image after adding noise and the medical image after preprocessing into the trained denoising network to obtain a segmentation result image of the skin lesion; The denoising network includes a denoising network encoder, a multi-level feature fusion module, a bottleneck structure and a denoising network decoder. The denoising network encoder includes five encoders whose parameters are not shared, each encoder includes four residual blocks and two linear attention modules, the multi-level feature fusion module includes ten fusion modules whose parameters are not shared, each fusion module includes a spatial attention module and a channel attention module, the bottleneck structure includes two residual blocks, a linear attention module and a self-attention module, and the denoising network decoder includes five decoders whose parameters are not shared, each decoder includes two residual blocks and a linear attention module; In a denoising network encoder, the medical image after adding noise and the preprocessed medical image are used as inputs, and are processed by a first encoder to obtain a first feature map, a second feature map, a first noise feature map, and a first medical image feature map; the first noise feature map and the first medical image feature map are input into a second encoder for processing to obtain a third feature map, a fourth feature map, a second noise feature map, and a second medical image feature map; the second noise feature map and the second medical image feature map are input into a third encoder for processing to obtain a fifth feature map, a sixth feature map, a third noise feature map, and a third medical image feature map; the third noise feature map and the third medical image feature map are input into a fourth encoder for processing to obtain a seventh feature map, an eighth feature map, a fourth noise feature map, and a fourth medical image feature map; the fourth noise feature map and the fourth medical image feature map are input into a fifth encoder for processing to obtain a ninth feature map, a tenth feature map, a fifth noise feature map, and a fifth medical image feature map; In the multi-level feature fusion module, the first feature map, the third feature map, the fifth feature map, the seventh feature map and the ninth feature map are used as the first input, and the second feature map, the fourth feature map, the sixth feature map, the eighth feature map and the tenth feature map are used as the second input. After the first fusion module processes the first input, a first multi-level fusion feature map is obtained. After the second fusion module processes the second input, a second multi-level fusion feature map is obtained. After the third fusion module processes the first input, a third multi-level fusion feature map is obtained. After the fourth fusion module processes the second input, a fourth multi-level fusion feature map is obtained. After the fifth fusion module processes the first input, a fifth multi-level fusion feature map is obtained. After the sixth fusion module processes the second input, a sixth multi-level fusion feature map is obtained. After the seventh fusion module processes the first input, a seventh multi-level fusion feature map is obtained. After the eighth fusion module processes the second input, an eighth multi-level fusion feature map is obtained. After the ninth fusion module processes the first input, a ninth multi-level fusion feature map is obtained. After the tenth fusion module processes the second input, a tenth multi-level fusion feature map is obtained. In the bottleneck structure, the fifth noise feature map and the fifth medical image feature map are used as inputs, and a sixth noise feature map is output after processing; In the denoising network decoder, the fifth decoder takes the sixth noise feature map, the tenth multi-level fusion feature map and the ninth multi-level fusion feature map as input, and after processing, obtains the seventh noise feature map. The fourth decoder takes the seventh noise feature map, the eighth multi-level fusion feature map and the seventh multi-level fusion feature map as input, and after processing, obtains the eighth noise feature map. The third decoder takes the eighth noise feature map, the sixth multi-level fusion feature map and the fifth multi-level fusion feature map as input, and after processing, obtains the ninth noise feature map. The second decoder takes the ninth noise feature map, the fourth multi-level fusion feature map and the third multi-level fusion feature map as input, and after processing, obtains the tenth noise feature map. The first decoder takes the tenth noise feature map, the second multi-level fusion feature map and the first multi-level fusion feature map as input, and after processing, obtains the segmentation result image of the skin lesions.

10. A computer electronic device, characterized in that: include: Image acquisition module, image segmentation module and result output module; An image acquisition module, used to acquire the preprocessed medical image required for the denoising network input and the medical image after adding noise; An image segmentation module, used to obtain a trained denoising network obtained by the multi-level feature fusion medical image segmentation method based on a diffusion model as described in any one of claims 1 to 8; The result output module is used to input the preprocessed medical image obtained by the image acquisition module and the medical image after adding noise into the denoising network obtained by the image segmentation module, and output the segmentation result image of the skin lesion.

Images