Image segmentation method and device based on diffusion model and generative adversarial network

Abstract

The present invention discloses an image segmentation method and device based on a diffusion model and a generative adversarial network, which are applied to the field of image processing technology. The method comprises: preprocessing an original image; training a diffusion model on the preprocessed image and outputting a predicted label image; training a discriminator model on the predicted label image and outputting a predicted classification label image; jointly training the diffusion model and the discriminator using a generative adversarial network, outputting parameters of the diffusion model and the discriminator model; segmenting the preprocessed image according to the parameters of the diffusion model and the discriminator model; avoiding the misleading of single information in the denoising process and improving the robustness of the model; splicing the original image and label data during the training process, which is more conducive to accurate blood vessel extraction; wherein the generative adversarial network trains the discriminator and the diffusion model, so that the network training converges faster and the segmentation accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image segmentation method and device based on a diffusion model and a generative adversarial network. Background Art

[0002] Time-of-flight magnetic resonance angiography (TMA) is a widely used imaging technique primarily used to examine vascular lesions in areas such as the carotid arteries, cerebral vessels, renal arteries, and lower limb vessels. It utilizes the distinct longitudinal magnetic field difference between saturated, stationary tissue and unsaturated, inflowing blood to generate images. When blood passes through a static magnetic field, the spins of hydrogen atoms within the bloodstream shift phase with the bloodstream's motion, resulting in variations in signal intensity. When the patient receives radiofrequency pulses from the magnetic field, the hydrogen atoms in the bloodstream emit signals, which are picked up by a receiving coil and processed and reconstructed by a computer to produce an image. Two imaging methods are available: two-dimensional (2D) time-of-flight imaging, which can only image a single vascular layer and cannot simultaneously image multiple layers. However, due to its high speed and high resolution, it can be used to detect intravascular thrombosis. Three-dimensional (3D) time-of-flight imaging can simultaneously image multiple layers, providing more comprehensive vascular information. However, due to its slower imaging speed and greater interference from factors such as respiration and motion, respiratory and motion control is required.

[0003] Currently, vascular segmentation techniques can be broadly categorized into two main categories: traditional methods based on graphics and statistics, and deep learning-based methods. Traditional graphics-based methods directly filter the original image, resulting in uneven grayscale distribution and range in the MRI image, and a small difference in grayscale values between the blood vessels and surrounding tissue. These methods, however, lack robustness. Deep learning methods, due to the localized nature of convolution operations, struggle to learn global and long-range semantic information.

[0004] In order to overcome these defects, this application proposes an image segmentation method and device based on a diffusion model and a generative adversarial network. Summary of the Invention

[0005] The purpose of this application is to provide an image segmentation method and device based on a diffusion model and a generative adversarial network, aiming to solve the problem of insufficient structural features in extracting subtle parts of blood vessels.

[0006] To achieve the above objectives, this application provides the following technical solutions:

[0007] This application provides an image segmentation method based on a diffusion model and a generative adversarial network, including:

[0008] Preprocess the original image;

[0009] Perform diffusion model training on the preprocessed image and output the predicted label image;

[0010] Performing discriminator model training on the predicted label image and outputting a predicted classification label image;

[0011] Using a generative adversarial network to jointly train the diffusion model and the discriminator, and outputting parameters of the diffusion model and parameters of the discriminator model;

[0012] The preprocessed image is segmented according to the parameters of the diffusion model and the parameters of the discriminator model.

[0013] Furthermore, the step of preprocessing the original image specifically includes the following steps:

[0014] The BET tool of FSL software was used to remove the skull in the original image;

[0015] Subtract the mean of all pixel values from the value of each pixel in the original image, adjust the mean of all pixel values to zero, and then divide the result by the standard deviation of the pixel values of the original image to adjust the discrete degree of the pixel values in the original image to be consistent; the specific formula is:

[0016] img=(img-img.mean) / (img.std+ε).

[0017] Furthermore, the step of training the diffusion model on the preprocessed image and outputting the predicted label image specifically includes the following steps:

[0018] The diffusion model includes a forward denoising process g and a reverse denoising process p;

[0019] The mean square error is used as the loss function to calculate the error between the real noise and the predicted noise.

[0020] Furthermore, the forward noise addition process specifically includes the following steps:

[0021] From the original image x t-1 to x t The formula of the forward noise addition process is expressed as:

[0022]

[0023] Where I is the identity matrix, β t is the variance of the forward process;

[0024] The original image x t-1 to x t Do t steps, then the formula is expressed as:

[0025]

[0026] The variable α t =1-β t and

[0027] Using reparameterization, x t Written as a function of x0:

[0028]

[0029] Combine the spliced image b of the original image with the label image x b Perform stitching to obtain an image stitched together with the original image and the label image The X dimension is (c+1, h, w), where is splicing in the channel dimension, c is the number of channels, h is the image height, and w is the image width; the forward noise formula is rewritten as:

[0030]

[0031] Furthermore, the reverse denoising process specifically includes the following steps:

[0032] Reverse denoising process p θ The Unet model parameter θ is learned as follows:

[0033]

[0034] According to x t Predict x t-1 :

[0035]

[0036] in σ t is the variance of model learning, z is the random component during sampling, and the model UNet input Noise∈ θ (x t , t) is learned by the model, the noise ∈ θ (x t , t) will be converted from x according to the above formula t minus;

[0037] Combine the spliced image b of the original image with the label image x b Perform stitching to obtain an image stitched together with the original image and the label image The X dimension is (c+1, h, w), where is splicing in the channel dimension, c is the number of channels, h is the image height, and w is the image width; the formula for reverse denoising is rewritten as:

[0038]

[0039] Furthermore, in the step of using the mean square error as the loss function to calculate the error between the real noise and the predicted noise, the following steps are specifically included:

[0040] Select a time point t∈[1, T] through the defined Diffusion class;

[0041] The noise image x at time t is obtained by the forward noise formula b，t and noise;

[0042] Input noisy image x b，t and t to the Unet network to calculate the predicted noise prednoise;

[0043] The mean square error formula is:

[0044]

[0045] Furthermore, the step of training a discriminator model for the predicted label image and outputting the predicted classification label image specifically includes the following steps:

[0046] Input the predicted label image and the manual label image of the original image;

[0047] The loss function is defined as binary cross entropy loss, the formula is:

[0048]

[0049] Where N is the number of samples in the data set; y i is the true label of the i-th sample, y i Usually 0 or 1; is the predicted output of the discriminator for the i-th sample, A real number between 0 and 1, indicating the probability that the sample belongs to the positive class; if the sample belongs to the positive class, then y i is 1, otherwise it is O;

[0050] When the true label is 1, the discriminator's predicted output is taken as the logarithm, and the result obtained represents the loss of the sample belonging to the positive class; when the true label is 0, the logarithm is inverted, and the result obtained represents the loss of the sample belonging to the negative class; and the loss of all samples is added up and divided by the number of samples to get the average loss;

[0051] The optimizer is used to calculate the gradient of the loss function, and the back-propagation algorithm is used to update the parameters of the discriminator and output the predicted classification label image.

[0052] Furthermore, the step of jointly training the diffusion model and the discriminator using a generative adversarial network and outputting the parameters of the diffusion model and the parameters of the discriminator model specifically includes the following steps:

[0053] The diffusion model is trained separately and the mean square error loss function is used to optimize the model Un e t, generate predicted segmentation mask;

[0054] The discriminator is trained separately and uses a binary cross entropy loss function to optimize the neural network and make judgments on the input image;

[0055] The diffusion model and the discriminator are jointly trained as follows:

[0056] Assume that the original image is x, the label of the original image is y, and the Gaussian noise standard deviation is G is the diffusion model, D is the discriminator;

[0057] Calculate the loss function of the diffusion model: L Diff =MSE(prednoise, noise);

[0058] Calculate the output probability and loss function of the discriminator: p = D(y), p' = D(y'), y i ∈{0, 1},

[0059] Calculate the joint loss function of the generative adversarial network: L G =log(1-p′), L D = -(log(p)+log(1-p′));

[0060] Output the diffusion model parameters θ1 and the discriminator parameters θ2, and use the back propagation algorithm to update the parameters θ1, θ2.

[0061] Furthermore, after the original image is preprocessed, the diffusion model is not trained. The preprocessed image, the manually labeled image and the Gaussian noise image are input into the diffusion model, and reverse sampling is performed using a recursive formula to obtain a predicted segmented image.

[0062] The present application provides an image segmentation device based on a diffusion model and a generative adversarial network, the device comprising:

[0063] Processing module: pre-process the original image;

[0064] Diffusion model training module: performs diffusion model training on preprocessed images and outputs predicted label images;

[0065] Discriminator training module: performs discriminator model training on the predicted label image and outputs the predicted classification label image;

[0066] Joint training module: using a generative adversarial network to jointly train the diffusion model and the discriminator, and outputting the parameters of the diffusion model and the parameters of the discriminator model;

[0067] Segmentation module: Segmenting the pre-processed image according to the parameters of the diffusion model and the parameters of the discriminator model.

[0068] The present application provides a device, which includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing an image segmentation method based on a diffusion model and a generative adversarial network; the processor is used to execute the program instructions stored in the memory to implement image segmentation based on the diffusion model and the generative adversarial network.

[0069] The present application provides a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute an image segmentation method based on a diffusion model and a generative adversarial network.

[0070] This application provides an image segmentation method and device based on a diffusion model and a generative adversarial network, which has the following beneficial effects:

[0071] This application improves the accuracy and stability of sampling by introducing original image information during the training and sampling process; a multi-layer attention mechanism is adopted in the process of training the preprocessed original image, and the original image and the image sampled in the previous step are combined through a splicing operation, thereby avoiding the misleading of single information in the denoising process, better fitting the grayscale gradation of the area of interest, and making the model more robust and accurate; by splicing the original image and the label image, the output of the network is guided towards the real manual label, with significant results; the model obtained by this application suppresses false positive areas and also has a good response to small blood vessel structures, making the resulting vascular network richer, thereby being able to capture the detailed features of tiny blood vessels. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flowchart of an image segmentation method based on a diffusion model and a generative adversarial network according to Example 1 of the present application;

[0073] Figure 2 This is a flowchart of an image segmentation method based on a diffusion model and a generative adversarial network according to Example 1 of the present application;

[0074] Figure 3The segmentation results of the TOF-MRA image before and after skull removal in Example 1 of the present application are shown;

[0075] Figure 4 This is the overall framework diagram of cerebral blood vessel segmentation in Example 1 of the present application;

[0076] Figure 5 The cross-section, sagittal, coronal, and labeled images of the MIDAS dataset and the New-data dataset in Example 1 of this application;

[0077] Figure 6 This is a schematic diagram of comparative experimental indicators of Example 1 of the present application;

[0078] Figure 7 This is a structural diagram of an image segmentation device based on a diffusion model and a generative adversarial network according to Example 2 of the present application;

[0079] Figure 8 A schematic diagram of the device structure provided in Example 3 of the present invention;

[0080] Figure 9 A schematic diagram of the storage medium structure provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0081] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0082] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0083] Example 1

[0084] See also Figure 1 and Figure 2 , respectively, are a flow chart and a flow chart of an image segmentation method based on a diffusion model and a generative adversarial network according to Example 1 of the present application; the specific steps include:

[0085] S1: Preprocess the original image.

[0086] In this example, the original TOF-MRA image contains not only brain tissue and its surface blood vessels, but also the skull, eyeballs, nose bridge, and air. These latter objects not only increase the computational complexity but can also mislead the extraction of cerebral blood vessels if not pre-processed. Therefore, to eliminate unnecessary information interference, the BET (Brain Extraction Tool) tool in the FSL (FMRIB's Software Library) software was used to remove the skull. The image fractional intensity threshold set in the software was manually adjusted, ultimately set to 0.2 as the ideal value.

[0087] See also Figure 3 , which are the segmentation results of the TOF-MRA image before and after skull removal in Example 1 of the present application; c1~c4 are the original images, in which the eyeballs and skull are the main interference factors; after software preprocessing, d1~d4 are the images after removing the skull, eyeballs and other parts.

[0088] In addition to preprocessing to remove the skull, the grayscale distribution of the image is also uneven, especially concentrated in the low grayscale area, so normalization is required. First, the value of each pixel in the original image is subtracted from the mean of all pixel values, and the average value of all pixel values is adjusted to zero. Then the result is divided by the standard deviation of the pixel values of the original image to adjust the discrete degree of the pixel values in the original image to be consistent; the specific formula is:

[0089] img=(img-img.mean) / (img.std+ε);

[0090] To prevent the standard deviation from being zero, division by zero is avoided, and a constant ε = 1e-8 is added to the formula; by standardizing the image, different images have the same pixel distribution, making it easier to compare their differences; and the distribution of pixel values in the image is closer to the normal distribution, which helps many machine learning models work better, while reducing the difficulty of model training and improving the convergence speed and accuracy of the model.

[0091] After preprocessing the original image, without performing diffusion model training, the preprocessed image, the manually labeled image and Gaussian noise are input into the diffusion model, and reverse sampling is performed using a recursive formula to obtain a predicted segmentation image.

[0092] S2: Perform diffusion model training on the preprocessed image and output the predicted label image.

[0093] In this embodiment, the steps of training the diffusion model on the pre-processed image and outputting the predicted label image specifically include the following steps S21 to S22 , and the implementation of each step is described in detail below.

[0094] S21: The diffusion model includes a forward denoising process q and a backward denoising process p.

[0095] The forward noise addition process specifically includes the following steps:

[0096] From the original image x t-1 to x t The formula of the forward noise addition process is expressed as:

[0097]

[0098] Where I is the identity matrix, β t is the variance of the forward process;

[0099] In each step, Gaussian noise is added to the image. This is done for t steps, and the formula is expressed as:

[0100]

[0101] The variable α t =1-β t and

[0102] Using reparameterization, x t Written as a function of x0:

[0103]

[0104] Combine the spliced image b of the original image with the label image x b Perform stitching to obtain an image stitched together with the original image and the label image The X dimension is (c+1, h, w), where is splicing in the channel dimension, c is the number of channels, h is the image height, and w is the image width; the forward noise formula is rewritten as:

[0105]

[0106] In the noise addition stage of the diffusion model, it is known that β t is the variance of the forward noise process; after experiments, it was found that the previous stable diffusion images mainly used β constructed based on the sin sine function. t , but it has been proved through theory and experiment that if the β constructed based on the coS cosine function is used t , will speed up the sampling process and ultimately reduce the loss calculation error; the formula is as follows:

[0107]

[0108]

[0109] The specific code structure is shown in the figure below:

[0110] def cosine_beta_schedule(timesteps, s=0.008):

[0111] ″″″

[0112] cosine schedule

[0113] steps = timesteps + 1

[0114] x=torch.1inspace(0, timesteps, steps, dtype=torch.float64)

[0115] alphas_cumprod=torch.cos(((x / timesteps)+s) / (1+s)*math.pi*0.5)**2

[0116] alphas_cumprod=alphas_cumprod / alphas_cumprod[0]

[0117] betas=1-(alphas_cumprod[1:] / alphas_cumprod[:-1])

[0118] return torch.clip(betas, 0, 0.999)

[0119] The reverse denoising process specifically includes the following steps:

[0120] Reverse denoising process p θ The Unet model parameter θ is learned as follows:

[0121]

[0122] According to x t Predict x t-1 :

[0123]

[0124] where σ t is the variance of model learning, z is the random component during sampling, and the model UNet input Noise∈ θ (x t , t) is learned by the model, the noise ∈ θ (x t , t) will be converted from x according to the above formulat minus;

[0125] Combine the spliced image b of the original image with the label image x b Perform stitching to obtain an image stitched together with the original image and the label image The X dimension is (c+1, h, w), where is splicing in the channel dimension, c is the number of channels, h is the image height, and w is the image width; the formula for reverse denoising is rewritten as:

[0126]

[0127] S22: Use mean square error as the loss function to calculate the error between the real noise and the predicted noise.

[0128] The training process of the diffusion model follows the conventional diffusion model strategy, that is, for each round of training, first randomly select a time point t∈[1, T] through the defined Diffusion class; then use the forward noise formula to obtain the noise image x at time t b，t and noise noise; input noise image x b，t The predicted noise prednoise is calculated by passing t to the Unet network. The mean squared error (MSE) is used as the loss function to calculate the error between the actual noise and the predicted noise prednoise. Through backpropagation and gradient descent, an excellent model is obtained and its parameters are saved for use during testing.

[0129] The formula for mean square error is:

[0130]

[0131] S3: Performing discriminator model training on the predicted label image and outputting the predicted classification label image.

[0132] In this embodiment, there are two types of input: one is the predicted label data generated by the diffusion model, and the other is the manually annotated image from the original image; the loss function is defined as the binary cross entropy loss, and the formula is:

[0133]

[0134] Where N is the number of samples in the data set; y i is the true label of the i-th sample, y i Usually 0 or 1; is the predicted output of the discriminator for the i-th sample.

[0135] In a binary classification problem, y i Usually 0 or 1, A real number between 0 and 1, indicating the probability that the sample belongs to the positive class; if the sample belongs to the positive class, then y i is 1, otherwise it is 0; when the true label is 1, the discriminator's predicted output is taken logarithm, and the result obtained represents the loss of the sample belonging to the positive class; when the true label is 0, the logarithm is inverted, and the result obtained represents the loss of the sample belonging to the negative class; and the losses of all samples are added up and divided by the number of samples to get the average loss.

[0136] After defining the loss function, we need to define an optimizer to update the parameters of the discriminator to minimize the loss function. The Adam optimizer is used.

[0137] After defining the input data, loss function, and optimizer, we can start training the discriminator. During the training process, the discriminator will receive input data and classify it using a neural network; the optimizer will calculate the gradient of the loss function and use the backpropagation algorithm to update the discriminator's parameters to make it classify the data more accurately.

[0138] S4: jointly train the diffusion model and the discriminator using a generative adversarial network, and output parameters of the diffusion model and parameters of the discriminator model.

[0139] See also Figure 4 , which is the overall framework diagram of cerebral blood vessel segmentation in Example 1 of the present application;

[0140] First, the diffusion model is trained separately, and the mean square error loss function is used to optimize the model Unet to generate the predicted segmentation mask;

[0141] The discriminator is then trained separately, using a binary cross-entropy loss function to optimize the neural network to the point where it can accurately distinguish whether the input image is a hand-labeled real image or an image synthesized by the diffusion model.

[0142] Finally, the diffusion model and the discriminator are jointly trained using a generative adversarial network, specifically:

[0143] Assume that the original image is x, the label of the original image is y, and the Gaussian noise standard deviation is G is the diffusion model, D is the discriminator;

[0144] Calculate the loss function of the diffusion model: L Diff =MSE(prednoise, noise);

[0145] Calculate the output probability and loss function of the discriminator: p = D(y), p' = D(y'), y i ∈{0, 1},

[0146] Calculate the joint loss function of the generative adversarial network: L G =log(1-p′), L D = -(log(p)+log(1-p′));

[0147] Output the diffusion model parameters θ1 and the discriminator parameters θ2, and use the back propagation algorithm to update the parameters θ1, θ2.

[0148] See also Figure 5 , are the cross-section, sagittal, coronal, and labeled images of the MIDAS dataset and the New-data dataset in Example 1 of the present application;

[0149] To demonstrate the superiority of the proposed overall segmentation framework combining the diffusion model and generative adversarial network, comparative experiments primarily used two datasets. These datasets were obtained from different regions and devices, resulting in different imaging parameters, which also indirectly demonstrate the scalability of the proposed model. The detailed parameters of the two datasets are as follows.

[0150] MIDAS: This dataset is publicly available and was released by Bullitt et al. at the CASILab at the University of North Carolina at Chapel Hill. It contains multimodal MR data from hundreds of volunteers, including DTI, time-of-flight (TOF)-MRA, T1, and T2 images. There are 109 TOF-MRA data sets, 42 of which have vascular markers. 30 of these were selected for training, and the remaining 12 for testing. The imaging equipment used was a Siemens ALLEGRA 3.0T MRI scanner. The imaging parameters were: TR = 35.0, TE = 3.56, FA = 22. The data space size was 448 × 448 × 128 voxels, with a voxel spacing of 0.51 × 0.51 × 0.80 mm.

[0151] New-data: This dataset is private and provided by the General Hospital of the Southern Theater Command of the People's Liberation Army of China. It contains 45 normal TOF-MRA data sets, 30 of which were selected for training and the remaining 15 for testing. The imaging equipment used was a GE SignaHDx 3.0T MRI scanner. The imaging parameters were TR = 25.0, TE = 3.5, and FA = 20. The data space size was 448 × 448 × 128 voxels, with a voxel spacing of 0.56 × 0.56 × 0.55 mm.

[0152] See also Figure 6 , which is a schematic diagram of comparative experimental indicators of Example 1 of the present application; in order to demonstrate the existing different methods and the proposed method through comparative experiments, a digital display of the experimental results was performed.

[0153] The comparative experiment evaluation metrics in the table include Dice score, HD95, and ASD, which can be used to evaluate model performance based on overlap, Huffman distance, and average surface distance. The 3D rendering clearly demonstrates that the proposed method can effectively segment blood vessels with consistent continuity, ensuring vessel integrity and detailed control.

[0154] In summary, Example 1 of the present application improves the accuracy and stability of sampling by introducing original image information during the training and sampling process; adopts a multi-layer attention mechanism and combines the original image and the image sampled in the previous step through a splicing operation, thereby avoiding the misleading of single information in the denoising process and improving the robustness of the model; during the training process, the original image and label data are spliced, so that the diffusion model does not sample the denoising of the original image, but samples the denoising of the segmented label image, which is more conducive to accurate blood vessel extraction; wherein the generative adversarial network trains the discriminator and the diffusion model, so that the network training accelerates convergence and the segmentation accuracy is improved.

[0155] Example 2

[0156] See also Figure 7 , which is a structural diagram of an image segmentation device based on a diffusion model and a generative adversarial network according to Example 2 of the present application; the specific contents include:

[0157] Processing module: pre-process the original image;

[0158] Diffusion model training module: performs diffusion model training on preprocessed images and outputs predicted label images;

[0159] Discriminator training module: performs discriminator model training on the predicted label image and outputs the predicted classification label image;

[0160] Joint training module: using a generative adversarial network to jointly train the diffusion model and the discriminator, and outputting the parameters of the diffusion model and the parameters of the discriminator model;

[0161] Segmentation module: Segmenting the pre-processed image according to the parameters of the diffusion model and the parameters of the discriminator model.

[0162] In summary, in Example 2 of the present application, the original image is preprocessed and then input into the diffusion model training to obtain a predicted label image, and then the discriminator judges the predicted label image and the true label image. Finally, a generative adversarial network is used to jointly train the diffusion model and the discriminator, so that the network training converges faster and the segmentation accuracy is further improved. In the diffusion model training stage, a multi-layer attention mechanism is used to splice the original image, the label image, and the noisy image, so that the diffusion model does not denoise the original image, but denoises the segmented label image, which is more conducive to accurate blood vessel extraction.

[0163] Example 3

[0164] See also Figure 8 , which is a schematic diagram of the device structure of an embodiment of the present application. The device 50 includes a processor 51 and a memory 52 coupled to the processor 51.

[0165] The memory 52 stores program instructions for implementing the above-mentioned image segmentation method based on the diffusion model and the generative adversarial network.

[0166] The processor 51 is configured to execute program instructions stored in the memory 52 to implement image segmentation based on the diffusion model and the generative adversarial network.

[0167] The processor 51 may also be referred to as a CPU (Central Processing Unit).

[0168] Processor 51 may be an integrated circuit chip with signal processing capabilities. Processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor.

[0169] Example 4

[0170] See also Figure 9, which is a structural diagram of the storage medium of the embodiment of the present application. The storage medium of the embodiment of the present application stores a program file 61 that can implement all the above methods, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer, server, mobile phone, tablet and other devices.

[0171] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0172] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

[0173] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

[0174] Of course, the present invention may have many other implementations. Based on this implementation, other implementations obtained by ordinary technicians in this field without any creative work are all within the scope of protection of the present invention.

Claims

1. An image segmentation method based on a diffusion model and a generative adversarial network, characterized in that: include: Preprocess the original image; Perform diffusion model training on the preprocessed image and output the predicted label image; Performing discriminator model training on the predicted label image and outputting a predicted classification label image; Using a generative adversarial network to jointly train the diffusion model and the discriminator, and outputting parameters of the diffusion model and parameters of the discriminator model; Segmenting the preprocessed image according to the parameters of the diffusion model and the parameters of the discriminator model; The steps of training the diffusion model on the preprocessed image and outputting the predicted label image specifically include the following steps: The diffusion model includes a forward denoising process q and a reverse denoising process p; The mean square error is used as the loss function to calculate the error between the real noise and the predicted noise; The forward noise addition process specifically includes the following steps: From the original image x t-1 to x t The formula of the forward noise addition process is expressed as: Where I is the identity matrix, β t is the variance of the forward process; The original image x t-1 to x t Do t steps, then the formula is expressed as: The variable α t =1-β t and Using reparameterization, x t Written as a function of x0: Combine the spliced image b of the original image with the label image x b Perform stitching to obtain an image stitched together with the original image and the label image The X dimension is (c+1,h,w), where is splicing in the channel dimension, c is the number of channels, h is the image height, and w is the image width; the forward noise formula is rewritten as:

2. The image segmentation method based on diffusion model and generative adversarial network according to claim 1, characterized in that: The steps of preprocessing the original image specifically include the following steps: The BET tool of FSL software was used to remove the skull in the original image; Subtract the mean of all pixel values from the value of each pixel in the original image, adjust the mean of all pixel values to zero, and then divide the result by the standard deviation of the pixel values of the original image to adjust the discrete degree of the pixel values in the original image to be consistent; the specific formula is: img=(img-img.mean) / (img.std+ε).

3. The image segmentation method based on diffusion model and generative adversarial network according to claim 1, characterized in that: The reverse denoising process specifically includes the following steps: Reverse denoising process p θ The Unet model parameter θ is learned as follows: According to x t Predict x t-1 : where σ t is the variance of model learning, z is the random component during sampling, and the model UNet input Noise∈ θ (x t ,t) is learned by the model, the noise ∈ θ (x t ,t) will be converted from x according to the above formula t minus; Combine the spliced image b of the original image with the label image x b Perform stitching to obtain an image stitched together with the original image and the label image The X dimension is (c+1,h,w), where is splicing in the channel dimension, c is the number of channels, h is the image height, and w is the image width; the formula for reverse denoising is rewritten as:

4. The image segmentation method based on diffusion model and generative adversarial network according to claim 1, characterized in that: In the step of using the mean square error as the loss function to calculate the error between the real noise and the predicted noise, the following steps are specifically included: Select a time point t∈[1,T] through the defined Diffusion class; The noise image x at time t is obtained by the forward noise formula b,t and noise; Input noisy image x b,t and t to the Unet network to calculate the predicted noise prednoise; The mean square error formula is:

5. The image segmentation method based on diffusion model and generative adversarial network according to claim 1, characterized in that: The step of training a discriminator model on the predicted label image and outputting the predicted classification label image specifically includes the following steps: Input the predicted label image and the manual label image of the original image; The loss function is defined as binary cross entropy loss, the formula is: Where N is the number of samples in the data set; y i is the true label of the i-th sample, y i Usually 0 or 1; is the predicted output of the discriminator for the i-th sample, A real number between 0 and 1, indicating the probability that the sample belongs to the positive class; if the sample belongs to the positive class, then y i is 1, otherwise 0; When the true label is 1, the logarithm of the discriminator's predicted output is taken, and the result obtained represents the loss of the sample belonging to the positive class; when the true label is 0, the logarithm is inverted, and the result obtained represents the loss of the sample belonging to the negative class; And add up the losses of all samples and divide by the number of samples to get the average loss; The optimizer is used to calculate the gradient of the loss function, and the back-propagation algorithm is used to update the parameters of the discriminator and output the predicted classification label image.

6. The image segmentation method based on diffusion model and generative adversarial network according to claim 1, characterized in that: The step of jointly training the diffusion model and the discriminator using a generative adversarial network and outputting the parameters of the diffusion model and the parameters of the discriminator model specifically includes the following steps: The diffusion model is trained separately, and the model Unet is optimized using the mean square error loss function to generate a predicted segmentation mask; The discriminator is trained separately and uses a binary cross entropy loss function to optimize the neural network and make judgments on the input image; The diffusion model and the discriminator are jointly trained as follows: Assume that the original image is x, the label of the original image is y, and the Gaussian noise standard deviation is G is the diffusion model, D is the discriminator; Calculate the loss function of the diffusion model: L Diff =MSE(prednoise,noise); Calculate the output probability and loss function of the discriminator: p = D(y), p' = D(y'), Calculate the joint loss function of the generative adversarial network: L G =log(1-p′),L D = -(log(p)+log(1-p′)); Output the diffusion model parameters θ1 and the discriminator parameters θ2, and use the back propagation algorithm to update the parameters θ1, θ2.

7. The image segmentation method based on diffusion model and generative adversarial network according to claim 1, characterized in that: After the original image is preprocessed, the diffusion model training is not performed. The preprocessed image, the manual label image and the Gaussian noise image are input into the diffusion model, and reverse sampling is performed using a recursive formula to obtain a predicted segmentation image.

8. A device for an image segmentation method based on a diffusion model and a generative adversarial network according to any one of claims 1 to 7, characterized in that: The device comprises: Processing module: pre-process the original image; Diffusion model training module: performs diffusion model training on preprocessed images and outputs predicted label images; Discriminator training module: performs discriminator model training on the predicted label image and outputs the predicted classification label image; Joint training module: using a generative adversarial network to jointly train the diffusion model and the discriminator, and outputting the parameters of the diffusion model and the parameters of the discriminator model; Segmentation module: Segmenting the pre-processed image according to the parameters of the diffusion model and the parameters of the discriminator model.

9. A device, characterized in that The device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing the image segmentation method based on a diffusion model and a generative adversarial network as described in any one of claims 1 to 7; and the processor is used to execute the program instructions stored in the memory to implement image segmentation based on the diffusion model and the generative adversarial network.

10. A storage medium, characterized in that: Program instructions executable by a processor are stored, and the program instructions are used to execute the image segmentation method based on a diffusion model and a generative adversarial network as described in any one of claims 1 to 7.

Images