A medical image generation method based on adversarial probability diffusion model

Abstract

A medical image generation method based on an adversarial probability diffusion model belongs to the field of image generation technology. The present invention solves the problems of low image generation efficiency, low definition and inability to ensure authenticity in existing medical image generation methods. The present invention specifically comprises: step S1, respectively counting the number of pixels occupied by the lesion area in each diseased medical image; step S2, respectively generating a change sequence of the number of pixels occupied by the lesion area for each diseased medical image; step S3, constructing a medical image generation model, and training the medical image generation model using each diseased medical image and the change sequence of the number of pixels occupied by the lesion area generated for each diseased medical image; step S4, using the healthy medical image as the input of the adversarial probability diffusion module of the trained medical image generation model to generate a diseased medical image. The present invention can be applied to the field of image generation technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of image generation, and in particular relates to a medical image generation method based on an adversarial probability diffusion model. Background Art

[0002] Deep learning has achieved significant success in multiple application areas such as computer vision, natural language processing, and reinforcement learning. However, to develop high-precision machine learning and deep learning models, a large number of training samples that fully cover the diversity of the population are often required. However, data availability in the field of medical images is very limited, and there are many reasons for this problem, such as the high cost of image acquisition, protection of sensitive patient information, limited number of disease cases, difficulty in data labeling, and differences in the location, scale, and appearance of abnormalities. Despite efforts to build large medical imaging datasets, other methods are limited except for using simple automatic methods, a lot of labor from radiologists, or mining from radiologists' reports. Therefore, generating medical image training samples is an effective alternative. However, in practice, various methods for generating medical images have a series of problems such as low efficiency of generating images, low clarity, and failure to ensure authenticity. Therefore, how to generate effective and sufficient medical data samples without or with limited expert participation remains a challenge. Summary of the invention

[0003] The purpose of the present invention is to solve the problems of low image generation efficiency, low definition and inability to ensure authenticity in existing medical image generation methods, and to propose a medical image generation method based on an adversarial probability diffusion model.

[0004] The technical solution adopted by the present invention to solve the above technical problems is:

[0005] A medical image generation method based on an adversarial probability diffusion model, the method specifically comprising the following steps:

[0006] Step S1, pre-processing each of the diseased medical images, that is, counting the number of pixels occupied by the lesion area in each of the diseased medical images;

[0007] Step S2: for any diseased medical image x in step S1 0 , the number of pixels occupied by the lesion area in the diseased medical image is recorded as C′ 0 , and generate a change sequence C = {C′ 1 ,…,C′ t ,…,C′ T};

[0008] Similarly, each diseased medical image is processed separately, that is, a change sequence of the number of pixels occupied by the lesion area is generated for each diseased medical image;

[0009] Step S3, constructing a medical image generation model including an adversarial probability diffusion module, a segmentation network and a classifier, and training the medical image generation model using each diseased medical image and a pixel number change sequence of the lesion area generated for each diseased medical image;

[0010] Step S4: Using the healthy medical image as the input of the adversarial probability diffusion module of the trained medical image generation model, the input healthy medical image is reversely transmitted in the adversarial probability diffusion module to generate a diseased medical image.

[0011] Furthermore, the training process of the medical image generation model is:

[0012] Step S301: for a diseased medical image x 0 , the diseased medical image x 0 and C′ 1 As input to the adversarial probability diffusion module of the medical image generation model;

[0013] Step S302: In the adversarial probability diffusion module, the diseased medical image x 0 As the input of the noise adding unit, the noise image output by the noise adding unit is obtained Then the noise image As a noise decoder M decoder The input of the noise decoder M decoder Generate medical image x 1 ;

[0014] And the medical image x 1 and the real diseased medical image x as the input of the discriminator, x∈X, X is the real diseased medical image set, and the loss function used by the discriminator is

[0015] The medical image x 1 As the image encoder M encoder The input is passed through the image encoder M encoder Output encoded image Using Encoded Images and noisy image Calculating Distribution Loss

[0016] The generated encoded image As the input of the denoising unit, the denoising unit generates a pair x 0 The restored image is denoted as x′ 0; Using the restored image x′ 0 and diseased medical images x 0 Calculate the reconstruction loss

[0017] Step S303: convert the medical image x 1 As the segmentation network M seg The input is passed through the segmentation network M seg Output medical image x 1 The lesion segmentation result S 1 , segmentation network M seg The loss function used is

[0018] Step S304: segment the lesion 1 As a medical image x 1 The label of the medical image x 1 and C′ 2 As the input of the adversarial probability diffusion module, and return to the process of executing step S302;

[0019] And so on, until the medical image x output by the adversarial probability diffusion module is obtained T And the segmentation network outputs the medical image x T The corresponding label;

[0020] The medical image x T As the input of the classifier, the classification result output by the classifier is used to calculate the classifier loss function;

[0021] Step S305: Similarly, steps S201 to S204 are simultaneously executed for each diseased medical image.

[0022] Furthermore, the classifier, discriminator, image encoder M encoder Sum Noise Decoder M decoder The CNN network is used.

[0023] Furthermore, the loss function for:

[0024]

[0025] Where k represents the kth classified image; K represents the number of images to be classified; y k Indicates whether the kth classified image is a real diseased medical image. If the kth classified image is a real diseased medical image, then y k is 1, if the kth classified image is not a real diseased medical image, then y k is 0; It represents the probability that the kth classified image is a real diseased medical image.

[0026] Furthermore, the distribution loss for:

[0027]

[0028] Among them, D KL represents the KL divergence loss.

[0029] Furthermore, the reconstruction loss is:

[0030]

[0031] Furthermore, the segmentation network is a U-Net network;

[0032] The segmentation network is pre-trained using a set of diseased medical images X and the corresponding segmentation labels Y.

[0033] Furthermore, the loss function for:

[0034]

[0035] in, represents background loss; Indicates prospect loss; Represents the lesion area size loss; α represents the weight of background loss in the generation loss; β represents the weight of foreground loss in the generation loss; γ represents the weight of lesion area size loss in the generation loss;

[0036]

[0037]

[0038]

[0039] in, According to the segmentation result S 1 For medical images x 1 The background area image obtained by division, According to the label S 0 For medical images x 0 The divided background area image, S 0 Represents the original image x 0 The segmentation label, C i Represents a medical image x 1 The actual number of pixels occupied by the lesion area.

[0040] Furthermore, the specific process of step S4 is as follows:

[0041] In the adversarial probability diffusion module, the input healthy medical image y T After the image encoder M encoder , and then the image encoder M encoder As the input of the denoising unit, and the output y of the denoising unit T-1 It is used as the input of the adversarial probability diffusion module again, and so on, until the output y of the denoising unit is obtained. 0 , that is, obtaining diseased medical images.

[0042] The beneficial effects of the present invention are:

[0043] The medical image generation method based on the adversarial probability diffusion model of the present invention generates a medical image with high definition by adversarial probability diffusion model, and significantly improves the efficiency of image generation. In addition, by gradually generating lesions, the authenticity of medical image generation is ensured, and the disease development process is evolved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of a medical image generation method based on an adversarial probability diffusion model of the present invention;

[0045] Figure 2 It is a structural diagram of the adversarial probability diffusion model. DETAILED DESCRIPTION

[0046] Specific implementation method 1: Combination Figure 1 The present embodiment describes a medical image generation method based on an adversarial probability diffusion model, and the method specifically comprises the following steps:

[0047] Step S1, pre-processing each of the diseased medical images, that is, counting the number of pixels occupied by the lesion area in each of the diseased medical images;

[0048] Step S2: for any diseased medical image x in step S1 0 , the number of pixels occupied by the lesion area in the diseased medical image is recorded as C′ 0 , and generate a change sequence C = {C′ 1 ,…,C′ t ,…,C′ T}; and C′ 1 ,…,C′ t ,…,C′ T The value of is gradually reduced until C′ T =0;

[0049] Similarly, each diseased medical image is processed separately, that is, a change sequence of the number of pixels occupied by the lesion area is generated for each diseased medical image;

[0050] Step S3, constructing a medical image generation model including an adversarial probability diffusion module, a segmentation network and a classifier, and training the medical image generation model using each diseased medical image and a pixel number change sequence of the lesion area generated for each diseased medical image;

[0051] Step S4: Using the healthy medical image as the input of the adversarial probability diffusion module of the trained medical image generation model, the input healthy medical image is reversely transmitted in the adversarial probability diffusion module to generate a diseased medical image.

[0052] Specific implementation method 2: Combination Figure 2 This embodiment is different from the first embodiment in that the training process of the medical image generation model is as follows:

[0053] Step S301: for a diseased medical image x 0 , the diseased medical image x 0 and C′ 1 As input to the adversarial probability diffusion module of the medical image generation model;

[0054] Step S302: In the adversarial probability diffusion module, the diseased medical image x 0 As the input of the noise adding unit (the processing method of the noise adding unit is: first generate random noise, and then add the generated random noise to the image), the noise image output by the noise adding unit is obtained Then the noise image As a noise decoder M decoder The input of the noise decoder M decoder Generate medical image x 1 (Generate medical image x 1 The goal is to make the number of pixels occupied by the lesion area C′ 1 );

[0055] And the medical image x 1 and the real diseased medical image x as the input of the discriminator (to distinguish between real images and generated images), x∈X, X is the set of real diseased medical images, and the loss function used by the discriminator is

[0056] The medical image x 1 As the image encoder M encoder The input is passed through the image encoder M encoder Output encoded image Using Encoded Images and noisy image Calculating Distribution Loss

[0057] The generated encoded image As the input of the denoising unit, the denoising unit (the network that the denoising unit can use includes but is not limited to the U-Net network) generates a pair x 0 The restored image is denoted as x′ 0 ; Using the restored image x′ 0 and diseased medical images x 0 Calculate the reconstruction loss

[0058] Step S303: convert the medical image x 1 As the segmentation network M seg The input is passed through the segmentation network M seg Output medical image x 1 The lesion segmentation result S 1 , segmentation network M seg The loss function used is

[0059] Step S304: segment the lesion 1 As a medical image x 1 The label of the medical image x 1 and C ′ 2 As the input of the adversarial probability diffusion module, and return to the process of executing step S302;

[0060] And so on, until the medical image x output by the adversarial probability diffusion module is obtained T And the segmentation network outputs the medical image x T The corresponding label;

[0061] The medical image x T As the input of the classifier (the classifier is a classifier trained using the healthy medical image set Z and the diseased medical image set X, and the trained classifier can accurately classify healthy images and diseased images), the classifier loss function is calculated through the classification results output by the classifier;

[0062] Step S305: Similarly, steps S201 to S204 are simultaneously executed for each diseased medical image.

[0063] The other steps and parameters are the same as those in the first embodiment.

[0064] Specific implementation method three: This implementation method is different from specific implementation method one or two in that the classifier, discriminator, image encoder M encoder Sum Noise Decoder Mdecoder The CNN network is used.

[0065] The other steps and parameters are the same as those in the first or second embodiment.

[0066] The image encoder M of the present invention encoder Sum Noise Decoder M decoder The networks that can be used include but are not limited to CNN networks.

[0067] Specific implementation method 4: This implementation method is different from any one of the specific implementation methods 1 to 3 in that the loss function for:

[0068]

[0069] Where k represents the kth classified image; K represents the number of images to be classified; y k Indicates whether the kth classified image is a real diseased medical image. If the kth classified image is a real diseased medical image, then y k is 1, if the kth classified image is not a real diseased medical image, then y k is 0; It represents the probability that the kth classified image is a real diseased medical image.

[0070] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0071] Specific implementation method 5: This implementation method is different from the specific implementation methods 1 to 4 in that the distribution loss for:

[0072]

[0073] Among them, D KL represents the KL divergence loss.

[0074] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0075] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that the reconstruction loss is:

[0076]

[0077] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0078] Specific implementation method 7: This implementation method is different from any one of specific implementation methods 1 to 6 in that the segmentation network is a U-Net network;

[0079] The segmentation network is pre-trained using a set of diseased medical images X and the corresponding segmentation labels Y.

[0080] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0081] The network that can be used in the segmentation network of the present invention includes a U-Net network, but is not limited to a U-Net network.

[0082] Specific implementation eight: This implementation differs from one of the specific implementations one to seven in that the loss function for:

[0083]

[0084] in, represents background loss; Indicates prospect loss; Represents the lesion area size loss; α represents the weight of background loss in the generation loss; β represents the weight of foreground loss in the generation loss; γ represents the weight of lesion area size loss in the generation loss;

[0085]

[0086]

[0087]

[0088] in, According to the segmentation result S 1 For medical images x 1 The background area image obtained by division, According to the label S 0 For medical images x 0 The divided background area image, S 0 Represents the original image x 0 The segmentation label, C i Represents a medical image x 1 The actual number of pixels occupied by the lesion area.

[0089] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0090] When i-1=0, Represents the background area image in the original image; S i represents the segmentation result of the i-th image generated, S i-1 represents the segmentation result of the i-1th image generated. When i-1=0, S i-1 Represents the segmentation label of the original image; C i Represents a medical image xi The actual number of pixels occupied by the lesion area.

[0091] Specific implementation method 9: This implementation method is different from the specific implementation methods 1 to 8 in that the specific process of step S4 is as follows:

[0092] In the adversarial probability diffusion module, the input healthy medical image y T After the image encoder M encoder , and then the image encoder M encoder As the input of the denoising unit, and the output y of the denoising unit T-1 It is used as the input of the adversarial probability diffusion module again, and so on, until the output y of the denoising unit is obtained. 0 , that is, obtaining diseased medical images.

[0093] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0094] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A medical image generation method based on an adversarial probability diffusion model, characterized in that: The method specifically comprises the following steps: Step S1, pre-processing each of the diseased medical images, that is, counting the number of pixels occupied by the lesion area in each of the diseased medical images; Step S2: for any diseased medical image x0 in step S1, the number of pixels occupied by the lesion area in the diseased medical image is recorded as C'0, and a change sequence of the number of pixels occupied by the lesion area is generated for the diseased medical image"' C={C1,…,C t ,…,C T }; Similarly, each diseased medical image is processed separately, that is, a change sequence of the number of pixels occupied by the lesion area is generated for each diseased medical image; Step S3, constructing a medical image generation model including an adversarial probability diffusion module, a segmentation network and a classifier, and training the medical image generation model using each diseased medical image and a pixel number change sequence of the lesion area generated for each diseased medical image; The training process of the medical image generation model is: Step S301: for the diseased medical image x0, the diseased medical image x0 and C'1 are used as inputs of the adversarial probability diffusion module of the medical image generation model; Step S302: In the adversarial probability diffusion module, the diseased medical image x0 is used as the input of the noise adding unit to obtain the noise image output by the noise adding unit. Then the noise image As a noise decoder M decoder The input of the noise decoder M decoder Generate medical image x1; The medical image x1 and the real diseased medical image x are used as the input of the discriminator, x∈X, X is the real diseased medical image set, and the loss function used by the discriminator is The medical image x1 is used as the image encoder M encoder The input is passed through the image encoder M encoder Output encoded image Using Encoded Images and noisy image Calculating distribution loss The generated encoded image As the input of the denoising unit, the denoising unit generates a restored image of x0, and the restored image is recorded as x'0; the reconstruction loss is calculated using the restored image x'0 and the diseased medical image x0 Step S303: Use the medical image x1 as the segmentation network M seg The input is passed through the segmentation network M seg Output the lesion segmentation result S1 of the medical image x1, the segmentation network M seg The loss function used is Step S304: Use the lesion segmentation result S1 as the label of the medical image x1, and then use the labeled medical image x1 and C'2 as inputs of the adversarial probability diffusion module, and return to the process of executing step S302; And so on, until the medical image x output by the adversarial probability diffusion module is obtained T And the segmentation network outputs the medical image x T The corresponding label; Medical image x T As the input of the classifier, the classification result output by the classifier is used to calculate the classifier loss function; Step S305: Similarly, perform steps S201 to S204 for each diseased medical image simultaneously; Step S4: Using the healthy medical image as the input of the adversarial probability diffusion module of the trained medical image generation model, the input healthy medical image is reversely transmitted in the adversarial probability diffusion module to generate a diseased medical image.

2. The medical image generation method based on the adversarial probability diffusion model according to claim 1, characterized in that: The classifier, discriminator, image encoder M encoder Sum Noise Decoder M decoder The CNN network is used.

3. The medical image generation method based on the adversarial probability diffusion model according to claim 2, characterized in that: The loss function for: Where k represents the kth classified image; K represents the number of images to be classified; y k Indicates whether the kth classified image is a real diseased medical image. If the kth classified image is a real diseased medical image, then y k is 1, if the kth classified image is not a real diseased medical image, then y k is 0; It represents the probability that the kth classified image is a real diseased medical image.

4. The medical image generation method based on the adversarial probability diffusion model according to claim 3, characterized in that: The distribution loss for: Among them, D KL represents the KL divergence loss.

5. The medical image generation method based on the adversarial probability diffusion model according to claim 4, characterized in that: The reconstruction loss is:

6. The medical image generation method based on the adversarial probability diffusion model according to claim 5, characterized in that: The segmentation network is a U-Net network; The segmentation network is pre-trained using a set of diseased medical images X and the corresponding segmentation labels Y.

7. The medical image generation method based on the adversarial probability diffusion model according to claim 6, characterized in that: The loss function for: in, represents background loss; Indicates prospect loss; Represents the lesion area size loss; α represents the weight of background loss in the generation loss; β represents the weight of foreground loss in the generation loss; γ represents the weight of lesion area size loss in the generation loss; in, represents the background area image obtained by dividing the medical image x1 according to the segmentation result S1, It represents the background area image obtained by dividing the medical image x0 according to the label S0, S0 represents the segmentation label of the original image x0, and C1 represents the actual number of pixels occupied by the lesion area in the medical image x1.

8. The medical image generation method based on the adversarial probability diffusion model according to claim 7, characterized in that: The specific process of step S4 is as follows: In the adversarial probability diffusion module, the input healthy medical image y T After the image encoder M encoder , and then the image encoder M encoder As the input of the denoising unit, and the output y of the denoising unit T-1 It is used as the input of the adversarial probability diffusion module again, and so on, until the output y0 of the denoising unit is obtained, that is, the diseased medical image is obtained.

Images