Training method, application method and related system of microscopic image generation model
Through the diffusion generation adversarial model, the microscope images are subjected to multi-step Gaussian noise processing and adversarial training, which solves the problem of time and quality of microscope images in the prior art, and realizes efficient and high-quality microscope images generation, supporting Medical diagnosis and research.
Patent Information
- Application Number
- CN202411067089.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-06
AI Technical Summary
The existing microscopic image generation methods take a long time and the quality of generating microscopic images is insufficient, making it difficult to meet the needs of high-precision medical diagnosis, especially in complex contexts, to capture detailed information.
The diffusion generation adversarial model is used, and the original microscopic image is subjected to multi-step Gaussian noise processing, and the generator and discriminator are used for adversarial training, the denoising distribution probability is calculated, and the adversarial loss function is optimized to improve the model generation ability.
It significantly improves the quality and efficiency of microscopic image generation, and can efficiently generate a large number of high-quality microscopic images without damaging image category information, supporting clinical diagnosis and medical research.
Smart Images

Figure CN119090762A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of microscopic image generation, and in particular to a training method, an application method and a related system of a microscopic image generation model. Background Art
[0002] In the field of modern medical diagnosis, microscopic images, as an important type of medical imaging data, play an irreplaceable role in clinical diagnosis and pathological research. Especially in the diagnosis of parasitic diseases, doctors usually need to rely on microscopic images of parasites to identify pathogens and observe pathological changes. However, due to the complexity of the parasite life cycle, the scarcity of infection cases and the limitations of sampling conditions, high-quality parasite microscopic images are difficult to obtain. These images often need to be collected under specific experimental conditions by experienced professionals operating microscopes, and the integrity and representativeness of the samples must be guaranteed during the collection process, making the collection work extremely arduous and time-consuming. Therefore, the development of a method and system that can efficiently generate high-quality parasite microscopic images is of great significance for promoting medical research, improving diagnostic efficiency and medical education.
[0003] Although deep learning technology has been widely used in the field of microscopic image generation, existing methods still have some limitations in actual operation. Among them, the most significant problem is that the generation process takes a long time, and the quality of the generated microscopic images cannot fully meet the needs of high-precision medical diagnosis. Especially in the context of subtle, complex and changeable microscopic image features, existing methods often find it difficult to capture enough detailed information and cannot generate high-quality microscopic images, thus affecting their application value in clinical medicine.
[0004] Based on this, how to more efficiently generate higher quality microscopic images and improve the quality and efficiency of generating microscopic images has become a technical problem that needs to be urgently solved in this field. Summary of the invention
[0005] The purpose of this application is to provide a training method, application method and related system for a microscopic image generation model, which can improve the quality and efficiency of generating microscopic images.
[0006] To achieve the above objectives, this application provides the following solutions:
[0007] In a first aspect, the present application proposes a training method for a microscopic image generation model, comprising:
[0008] Acquire raw microscopic images.
[0009] Gaussian noise is added to the original microscopic image to obtain a first microscopic image.
[0010] According to the first microscopic image, a second microscopic image is generated using a generator in a microscopic image generation model; the microscopic image generation model is a model composed of a diffusion model and a generative adversarial model, and the generative adversarial model includes a generator and a discriminator.
[0011] Gaussian noise is added to the second microscopic image to obtain a third microscopic image; the Gaussian noise added to the third microscopic image and the first microscopic image are both multi-step Gaussian noise, and the Gaussian noise added to the first microscopic image is t-step Gaussian noise, and the Gaussian noise added to the third microscopic image is t-1 step Gaussian noise.
[0012] According to the third microscopic image and the first microscopic image, the probability that the third microscopic image is a reasonable denoising distribution of the first microscopic image is calculated using the discriminator in the microscopic image generation model.
[0013] The microscopic image generation model is trained according to the probability that the third microscopic image serves as a reasonable denoising distribution of the first microscopic image and the adversarial loss function to obtain a trained microscopic image generation model.
[0014] Optionally, the original microscopic image is processed by adding Gaussian noise to obtain a first microscopic image, which is specifically expressed as:
[0015] X t =U(X 0 ,t);
[0016] Among them, X t represents the first microscopic image, X 0 represents the original microscopic image, U(·) represents the Gaussian noise adding operation, t represents the number of noise adding steps, t∈[1,2,…,T].
[0017] Optionally, according to the first microscopic image, a second microscopic image is generated using a generator in a microscopic image generation model, which is specifically expressed as:
[0018]
[0019] in, represents the second microscopic image, X 0 represents the original microscopic image, G(·) represents the generator, X t represents the first microscopic image, Z represents a random Gaussian variable, t represents the number of noise addition steps, t∈[1,2,…,T];
[0020] Gaussian noise is added to the second microscopic image to obtain a third microscopic image, which is specifically expressed as:
[0021]
[0022] in, represents the third microscopic image, U(·) represents the Gaussian noise adding process, and t-1 represents the number of noise adding steps applied at t-1.
[0023] Optionally, the adversarial loss function is expressed as:
[0024]
[0025] in, represents the discriminator loss function, represents the generator loss function, Indicates the third microscopic image, X t-1 represents the microscopic image of the previous noise-added step of the first microscopic image, D(·) represents the discriminator, X t represents the first microscopic image, t represents the number of noise adding steps, t∈[1,2,…,T].
[0026] In a second aspect, the present application proposes an application method of a microscopic image generation model, comprising:
[0027] Determine a target image to be generated;
[0028] Random Gaussian noise with the same shape as the target image matrix to be generated is input into a trained microscopic image generation model to generate a new high-quality microscopic image; the trained microscopic image generation model is a model trained according to the training method for a microscopic image generation model described in the first aspect.
[0029] In a third aspect, the present application proposes a training system for a microscopic image generation model, comprising:
[0030] The original microscopic image acquisition module is used to acquire the original microscopic image.
[0031] The first noise adding module is used to add Gaussian noise to the original microscopic image to obtain a first microscopic image.
[0032] A generation module is used to generate a second microscopic image according to the first microscopic image using a generator in a microscopic image generation model; the microscopic image generation model is a model composed of a diffusion model and a generative adversarial model, and the generative adversarial model includes a generator and a discriminator.
[0033] The second noise adding module is used to add Gaussian noise to the second microscopic image to obtain a third microscopic image; the Gaussian noise added to the third microscopic image and the first microscopic image are both multi-step Gaussian noise, and the Gaussian noise added to the first microscopic image is t-step Gaussian noise, and the Gaussian noise added to the third microscopic image is t-1 step Gaussian noise.
[0034] The identification module is used to calculate the probability that the third microscopic image is a reasonable denoising distribution of the first microscopic image based on the third microscopic image and the first microscopic image and using the discriminator in the microscopic image generation model.
[0035] The model training module is used to train the microscopic image generation model according to the probability of the third microscopic image being a reasonable denoising distribution of the first microscopic image and the adversarial loss function to obtain a trained microscopic image generation model.
[0036] Optionally, the first noise adding module adds Gaussian noise to the original microscopic image to obtain a first microscopic image, which is specifically expressed as:
[0037] X t =U(X 0 ,t);
[0038] Among them, X t represents the first microscopic image, X 0 represents the original microscopic image, U(·) represents the Gaussian noise adding operation, t represents the number of noise adding steps, t∈[1,2,…,T].
[0039] Optionally, the generating module generates the second microscopic image according to the first microscopic image using a generator in a microscopic image generating model, which is specifically expressed as:
[0040]
[0041] in, represents the second microscopic image, X 0 represents the original microscopic image, G(·) represents the generator, X t represents the first microscopic image, Z represents a random Gaussian variable, t represents the number of noise addition steps, t∈[1,2,…,T];
[0042] Gaussian noise is added to the second microscopic image to obtain a third microscopic image, which is specifically expressed as:
[0043]
[0044] in, represents the third microscopic image, U(·) represents the Gaussian noise adding process, and t-1 represents the number of noise adding steps applied at t-1.
[0045] Optionally, the adversarial loss function is expressed as:
[0046]
[0047] in, represents the discriminator loss function, represents the generator loss function, Indicates the third microscopic image, X t-1 represents the microscopic image of the previous noise-added step of the first microscopic image, D(·) represents the discriminator, X t represents the first microscopic image, t represents the number of noise adding steps, t∈[1,2,…,T].
[0048] In a fourth aspect, the present application proposes an application system of a microscopic image generation model, comprising:
[0049] A target image determination module, used to determine a target image to be generated;
[0050] A model application module is used to input random Gaussian noise of the same shape as the target image matrix to be generated into a trained microscopic image generation model to generate a new high-quality microscopic image; the trained microscopic image generation model is a model trained according to the training method for a microscopic image generation model described in the first aspect.
[0051] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0052] The present application provides a training method, an application method and a related system for a microscopic image generation model, which introduces Gaussian noise into the original microscopic image to enrich the image expression. The generator in the microscopic image generation model is used to generate a second microscopic image based on the first microscopic image after adding noise. The generator is used to achieve the denoising effect on the first microscopic image after adding noise, and a large number of high-quality second microscopic images can be output. Then, the same Gaussian noise processing method is used to apply t-1 steps of Gaussian noise to the second microscopic image, thereby obtaining a third microscopic image after adding noise. The discriminator is used to calculate the probability that the third microscopic image is a reasonable denoising distribution of the first microscopic image based on the first microscopic image and the third microscopic image, thereby achieving the evaluation of the denoising effect of the microscopic image output by the generator by the discriminator. When training the microscopic image generation model, the adversarial loss optimization network is used to enhance the model's generation and differentiation capabilities through the process of the generator denoising the noisy microscopic image and the discriminator evaluating the generator's denoising effect, so that the trained microscopic image generation model can be used to generate high-quality microscopic image data of the same category on a large scale, effectively improving the quality and efficiency of the model's generation of microscopic images. This application can rely on a small number of real microscopic image samples to generate microscopic images in large quantities and efficiently without damaging the image category information, which is of great significance to clinical medical diagnosis and research. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0054] Figure 1 A flowchart of a method for training a microscopic image generation model provided in one embodiment of the present application.
[0055] Figure 2 A schematic diagram of the structure of a microscopic image generation model provided in one embodiment of the present application.
[0056] Figure 3 A schematic diagram of a training method for a microscopic image generation model provided in one embodiment of the present application.
[0057] Figure 4 A structural block diagram of a training system for a microscopic image generation model provided in one embodiment of the present application.
[0058] Figure 5 A structural block diagram of an application system of a microscopic image generation model provided in one embodiment of the present application. DETAILED DESCRIPTION
[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0060] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0061] Example 1
[0062] like Figure 1 As shown, this embodiment proposes a training method for a microscopic image generation model, which combines a diffusion model (Diffusion Model) with a generative adversarial network (GAN) to form a diffusion generative adversarial model (DGAM) as a microscopic image generation model. In other words, the microscopic image generation model in this embodiment uses a diffusion generative adversarial model. Figure 2As shown, it aims to efficiently generate high-quality microscopic images. Through this diffusion-generated adversarial model, the key features of microscopic images, such as cell morphology and structural details, can be quickly and accurately reproduced, effectively supporting medical research and diagnosis processes, and providing strong technical support for clinical and laboratory research. The training method specifically includes the following steps:
[0063] Step S1, obtaining an original microscopic image.
[0064] Step S2: Add Gaussian noise to the original microscopic image to obtain a first microscopic image.
[0065] Step S3, based on the first microscopic image, using the generator in the microscopic image generation model, generate a second microscopic image; the microscopic image generation model is a model composed of a diffusion model and a generative adversarial model, and the generative adversarial model includes a generator and a discriminator. Among them, the diffusion model is used to learn the information attenuation law caused by noise, and then use the learned denoising distribution to help the generative adversarial model restore the original microscopic image. The generative adversarial model constructs an adversarial training framework through a generator and a discriminator, and improves the performance of the generative adversarial model through the adversarial training process between the generator and the discriminator.
[0066] Step S4, adding Gaussian noise to the second microscopic image to obtain a third microscopic image; that is, applying t-1 step Gaussian noise to the generated second microscopic image to obtain a third microscopic image. Wherein, the Gaussian noise added to the third microscopic image and the first microscopic image are both multi-step Gaussian noise, and the first microscopic image is added with t-step Gaussian noise, and the third microscopic image is added with t-1 step Gaussian noise.
[0067] Step S5: According to the third microscopic image and the first microscopic image, using the discriminator in the microscopic image generation model, calculate the probability that the third microscopic image is a reasonable denoising distribution of the first microscopic image.
[0068] Step S6: According to the probability of the third microscopic image being a reasonable denoising distribution of the first microscopic image and the adversarial loss function, the microscopic image generation model is trained to obtain a trained microscopic image generation model. The trained microscopic image generation model is used to efficiently and in large quantities generate high-quality microscopic images with a style similar to that of the original microscopic image after inputting random Gaussian noise. The adversarial loss function includes the reasonable denoising probability of the third microscopic image being the first microscopic image and the reasonable denoising probability distribution of the microscopic image of the last noise step of the first microscopic image as the first microscopic image.
[0069] Figure 3The principle of the training method of the microscopic image generation model is shown, such as Figure 3 As shown, in this embodiment, when training the microscopic image generation model, the main technical principles are as follows:
[0070] Step 101: Implement multi-step Gaussian noise injection on the original microscopic image to obtain a first microscopic image, wherein the first microscopic image is a microscopic image after the original microscopic image is noised.
[0071] In this embodiment, the original microscopic image X 0 A multi-step noisy process based on Gaussian distribution is implemented to generate the noisy image X t and X t-1 The noise adding process can be expressed as formula (1):
[0072] X t =U(X 0 ,t) (1).
[0073] Among them, X t represents the first microscopic image, X 0 represents the original microscopic image, U(·) represents the operation of adding Gaussian noise, t represents the number of noise adding steps, t∈[1,2,…,T]. In each training cycle (epoch), the algorithm randomly selects a noise adding time step t, and performs noise adding processing on the microscopic image according to the predetermined Gaussian distribution, thereby obtaining the first microscopic image after noise adding. In other words, t randomly selects a value from [1,2,…T] in each epoch for auxiliary training. Specifically, given an original microscopic image x 0 ∈X 0 , the noise adding step can also be expressed by the following formula:
[0074]
[0075] Among them, x t represents the microscopic image after adding noise t time steps, β t is a hyperparameter, set to a sequence between [0,1] that increases linearly with time steps, and is used to control the amount of noise added to the microscopic image in each noise addition time step t.
[0076] Step 102: The generator outputs a denoised image to obtain a second microscopic image, wherein the second microscopic image is a microscopic image obtained by denoising the first microscopic image through the action of the generator.
[0077] In this embodiment, the microscopic image generation model can be divided into two modules: the generator G(·) and the discriminator D(·). In this step, the input of the generator G(·) includes the noise time step t, the random latent variable And the first microscopic image Xt The generator can adopt a U-Net structure or a Transformer structure, etc. In this embodiment, a U-Net structure with multiple ResNet blocks is preferred. At the same time, the generator of this embodiment draws on the design concept of StyleGAN and introduces latent variables sampled from a random Gaussian distribution. All the group normalization (GN) layers in the network are replaced with adaptive group normalization (AdaGN) layers to finely enhance the diversity of generated images at different levels, making the generated images not only high-quality but also rich and diverse in style. The calculation process of the generator can be expressed as:
[0078]
[0079] in, represents the microscopic image output by the generator, i.e., the second microscopic image, X 0 represents the original microscopic image, G(·) represents the generator, X t represents the first microscopic image, Z represents a random Gaussian variable, t represents the number of noise adding steps, t∈[1,2,…,T].
[0080] In this embodiment, when the generator is embedded in the network, the random variable First, it is converted into an embedding vector W through a linear layer, and then the model feature map and the embedding vector W are input into the AdaGN layer together. The detailed expression in the image generation process is dynamically adjusted through the following formula to enhance the diversity and authenticity of the generated image:
[0081]
[0082] Among them, m i represents the i-th channel of the model feature map of the input AdaGN layer, μ(·) and σ(·) are used to calculate the mean and variance respectively, (y s,i ,y b,i )∈Y represents the scaling and translation parameters used to transform the feature map, and the feature map transformation parameters Learning is performed by a single linear layer, where N l Represents the total number of channels of the feature map of the input model.
[0083] In this embodiment, in order to enable the model generator G(·) to fully consider the impact of the number of noise addition steps t on image generation, sine position embedding is used to transform t into a time encoding vector embedding generator G(·). The specific formula is as follows:
[0084]
[0085] Where PE(·) represents the function that maps time step t to a temporal encoding vector, j represents the position index of the encoding, and d is a hyperparameter representing the total length of the temporal encoding vector.
[0086] Step 103: applying t-1 steps of Gaussian noise to the generated second microscopic image to obtain a third microscopic image, wherein the third microscopic image is a microscopic image after the noise is added to the second microscopic image.
[0087] In this embodiment, the noise adding function U(·) mentioned in step 101 is used to generate the second microscopic image output by the generator. Add the Gaussian noise of step t-1 to get the third microscopic image The noise addition formula is similar to formula (1), as follows:
[0088]
[0089] in, represents the third microscopic image, represents the second microscopic image, U(·) represents the operation of adding Gaussian noise, t represents the number of noise adding steps, and t-1 represents the number of noise adding steps applied t-1. Here, t-1 steps of Gaussian noise with the same distribution as the original image are applied to the generated second microscopic image.
[0090] Step 104: The discriminator evaluates the denoising effect of the microscopic image.
[0091] In this embodiment, the noisy image and the denoised image are input to the discriminator D(·) together. After multi-layer CNN (Convolutional Neural Network) feature extraction, the output features are spliced and then passed into a classifier composed of multiple fully connected layers for evaluation, and the probability of the denoised image being a reasonable denoised distribution of the noisy image is calculated. The discriminator D(·) can adopt a CNN network structure or a Vision Transformer model structure, etc. In this embodiment, the discriminator D(·) preferably has a CNN network structure with residual connections similar to the generator G(·), and t is embedded in the discriminator D(·) in the same sinusoidal coding processing as the generator. The discriminator D(·) is gradually optimized through the loss function, thereby comprehensively analyzing the above inputs and improving its identification ability.
[0092] Step 105: Optimize the DGAM model using adversarial loss.
[0093] In this embodiment, the adversarial loss includes two losses: the discriminator loss and the generator loss. The goal of the discriminator D(·) is to identify X as much as possible. t Therefore, the discriminator loss function The discriminator D(·) is encouraged to identify the original microscopic image processed X t-1 To reasonably denoise the distribution, we also need to identify the is an unreasonable denoising distribution. Therefore, the discriminator loss function Defined as:
[0094]
[0095] in, represents the discriminator loss function, Indicates the third microscopic image, X t-1 represents the microscopic image of the previous noise-added step of the first microscopic image, D(·) represents the discriminator, X t represents the first microscopic image, t represents the number of noise adding steps, t∈[1,2,…,T].
[0096] In this embodiment, the goal of the generator G(·) is to generate an image that retains as many features of the original microscopic image as possible. The generator loss function is The generator G(·) is encouraged to create images that are difficult for the discriminator D(·) to distinguish. Therefore, the generator loss function is Defined as:
[0097]
[0098] in, represents the generator loss function, Indicates the third microscopic image, X t represents the first microscopic image, t represents the number of noise adding steps, t∈[1,2,…,T].
[0099] It should be noted that the formula of adversarial loss is similar to that of the generative adversarial loss of the GAN network. Both the generator and the discriminator need to train parameters based on the output of the discriminator. The training objective of the generator is opposite to that of the discriminator.
[0100] Finally, the overall adversarial loss function is expressed as a combination of the generator loss and the discriminator loss. When training the model, the discriminator and generator are alternately trained using the corresponding loss functions until the preset number of iterations is reached.
[0101] In this embodiment, the probability value obtained by the discriminator will be input into the loss function for model training. During training, the discriminator should be input with two sets of samples: the first microscopic image X t and the previous noise sample X of the first microscopic image t-1 As the first set of samples; the first microscopic image X t and the third microscopic image As the second set of samples. First, the previous step noise sample X of the first microscopic image t-1 As the first microscopic image X t The probability value of a reasonable denoising distribution is given by D(t,X t-1 ,X t ) indicates; secondly, the third microscopic image As the first microscopic image X t The probability value of a reasonable denoising distribution is given by the loss function The former is analogous to the probability of inputting a real image into the discriminator in the original GAN network. The discriminator should consider it as a real and reasonable denoising distribution, and the larger the probability value, the better. The latter is analogous to the probability of inputting a generated (Fake) image into the discriminator in the original GAN network. The discriminator should consider it as an unreasonable denoising distribution, and the smaller the probability value, the better. In this way, as the training progresses, the discriminator will gain a stronger ability to distinguish whether the denoising distribution is reasonable; in contrast, the loss of the generator encourages it to generate images that can confuse the discriminator more.
[0102] This embodiment optimizes the adversarial loss function, which not only improves the quality of generated microscopic images, but also ensures the authenticity and reliability of generated microscopic images.
[0103] In this embodiment, after optimizing the DGAM model against the loss and obtaining the trained DGAM model in step 105, the following steps may also be included:
[0104] Step 106: Generate a large amount of microscopic image data using the trained DGAM model.
[0105] In this embodiment, the trained DGAM model has the ability to efficiently generate microscopic images and can produce a large amount of high-quality microscopic image data in a short period of time. These microscopic image data can not only be used for in-depth research in clinical pathology, but also provide strong data support for disease diagnosis and treatment.
[0106] In this embodiment, given a set of noise O with the same size as the generated image sampled from a random Gaussian distribution and a fully trained generator G(·) in the DGAM model, high-quality microscopic images with a style similar to the original microscopic images can be generated in batches by the following formula:
[0107]
[0108] Among them, H(·) represents the process of cyclic denoising generation, represents a set of generated high-quality microscopic images, T is a pre-set hyperparameter, and O represents the random Gaussian noise input when generating the image.
[0109] It should be noted that, in this embodiment, in order to prevent the DGAM model from being unstable and causing mode collapse, It is not generated by the generator G(·) in a single step, but is generated by denoising from random Gaussian noise step by step through the following steps in a loop. That is, the cyclic denoising generation process represented by H(·) in formula (9) is as follows: given random Gaussian noise O, first use the well-trained generator G(·) to generate a set of restored images Then use formula (1) in step 101 to add T-1 steps of noise to the image to obtain the denoised image Then, repeat the above steps to obtain the denoised image Wait until the image is gradually restored to an image without noise Through the above-mentioned method of cyclic denoising to generate images, the DGAM model can decompose the generation process into a series of denoising steps, and each step is relatively simple, ensuring that the model has better stability and pattern coverage.
[0110] In the training process, the present embodiment adopts parasite or cell images of 1000 times magnification under a bright light microscope (Olympus IX53), a total of 5741 Toxoplasma samples, 6981 red blood cell samples and 5758 Plasmodium samples. Using the DGAM model, the microscopic images of different categories including Toxoplasma, red blood cells and Plasmodium are successfully batch-generated. In the generation process, the model not only accurately captures the key shapes and textures of the microscopic images, but also ensures the accuracy of the category attributes. The microscopic image generation technology based on the DGAM model ensures the high quality of the image under the premise of retaining biological significance. Therefore, the model can not only effectively generate microscopic images in batches, but also provide strong technical support for medical research, especially in the case of scarcity of microscopic image samples, and provides a new data acquisition method for scientific researchers, with high application value and broad development prospects.
[0111] In this embodiment, the FID (Fréchet Inception Distance) indicator is used as a tool to quantify the quality of the generated image, and the time required to generate a batch (100) of microscopic images on the A100 GPU is recorded to evaluate the generation efficiency of the model. The experimental results are shown in Table 1. The experimental results show that the microscopic image generation method and system based on the microscopic image generation model in this embodiment can efficiently generate hundreds of high-quality microscopic images in less than one second. The generated image is not only highly similar to the original microscopic image visually, but also shows extremely high consistency in key features. Therefore, the DGAM algorithm has important practical significance and application value for promoting medical research.
[0112] Table 1 Experimental results of the generation effect of DGAM model on microscopic images
[0113] Microscopic image category FID Time(s) Red blood cells 51.58 0.72
[0114] This embodiment introduces a new diffusion process and a targeted adversarial training strategy to propose a diffusion generative adversarial model, which can efficiently generate high-quality microscopic images. This model not only significantly reduces the time cost of image generation, but also improves the clarity and detail authenticity of the image, better preserves the biological characteristics of the original microscopic image, and has potential contributions in promoting the development of clinical medicine and improving the accuracy and efficiency of medical image analysis.
[0115] Example 2
[0116] This embodiment provides an application method of a microscopic image generation model. This embodiment is actually a method for actually applying the model based on the training method in Embodiment 1. The application method includes the following steps:
[0117] Determining the target images to be generated, including the size and number of the target images to be generated;
[0118] Random Gaussian noise with the same shape as the target image matrix to be generated is input into the trained microscopic image generation model to generate a new high-quality microscopic image; the trained microscopic image generation model is a model trained according to the training method for a microscopic image generation model described in Example 1.
[0119] Example 3
[0120] This embodiment provides a training system for a microscopic image generation model. This embodiment is actually a system corresponding to the training method of Embodiment 1. Figure 4 As shown, the training system specifically includes:
[0121] The original microscopic image acquisition module 201 is used to acquire the original microscopic image.
[0122] The first noise adding module 202 is used to add Gaussian noise to the original microscopic image to obtain a first microscopic image.
[0123] The generation module 203 is used to generate a second microscopic image according to the first microscopic image using a generator in a microscopic image generation model; the microscopic image generation model is a model composed of a diffusion model and a generative adversarial model, and the generative adversarial model includes a generator and a discriminator.
[0124] The second noise adding module 204 is used to add Gaussian noise to the second microscopic image to obtain a third microscopic image; the Gaussian noise added to the third microscopic image and the first microscopic image are both multi-step Gaussian noise, and the Gaussian noise added to the first microscopic image is t-step Gaussian noise, and the Gaussian noise added to the third microscopic image is t-1 step Gaussian noise.
[0125] The identification module 205 is used to calculate the probability that the third microscopic image is a reasonable denoised distribution of the first microscopic image based on the third microscopic image and the first microscopic image and using the discriminator in the microscopic image generation model.
[0126] The model training module 206 is used to train the microscopic image generation model according to the probability that the third microscopic image is a reasonable denoising distribution of the first microscopic image and the adversarial loss function to obtain a trained microscopic image generation model. The adversarial loss function includes the reasonable denoising probability of the third microscopic image as the first microscopic image and the reasonable denoising probability of the microscopic image at the last noise addition step of the first microscopic image as the first microscopic image.
[0127] This embodiment forms an efficient and reliable microscopic image generation system through the coordinated cooperation between the above modules. Gaussian noise is added to the original microscopic image data to provide necessary data disturbance for the generation process; then, based on the noise-processed data, a denoised microscopic image is generated using a deep learning algorithm to simulate the characteristics and details of a real microscopic image; then the image denoising effect is evaluated to ensure the rationality and accuracy of the microscopic image during the denoising process; then the DGAM model is trained through an adversarial loss function to enable it to generate high-quality microscopic images; and then a fully trained DGAM model is used to batch generate a large number of microscopic images to meet the application needs of clinical and scientific research. Not only does it improve the speed of image generation, but it also ensures the high quality and application value of the generated images.
[0128] Example 4
[0129] This embodiment provides an application system of a microscopic image generation model. This embodiment is actually a system corresponding to the application method of embodiment 2. Figure 5 As shown, the application system specifically includes:
[0130] The target image determination module 207 is used to determine the target image to be generated.
[0131] The model application module 208 is used to input random Gaussian noise of the same shape as the target image matrix to be generated into the trained microscopic image generation model to generate a new high-quality microscopic image; the trained microscopic image generation model is a model trained according to the training method of a microscopic image generation model described in Example 1.
[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0133] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0134] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A training method for a microscopic image generation model, characterized in that: include: Acquire original microscopic images; Adding Gaussian noise to the original microscopic image to obtain a first microscopic image; Generate a second microscopic image according to the first microscopic image using a generator in a microscopic image generation model; the microscopic image generation model is a model composed of a diffusion model and a generative adversarial model, and the generative adversarial model includes a generator and a discriminator; Adding Gaussian noise to the second microscopic image to obtain a third microscopic image; the Gaussian noise added to the third microscopic image and the first microscopic image are both multi-step Gaussian noise, and the Gaussian noise added to the first microscopic image is t-step Gaussian noise, and the Gaussian noise added to the third microscopic image is t-1-step Gaussian noise; According to the third microscopic image and the first microscopic image, using the discriminator in the microscopic image generation model, calculating the probability that the third microscopic image is a reasonable denoised distribution of the first microscopic image; The microscopic image generation model is trained according to the probability that the third microscopic image serves as a reasonable denoising distribution of the first microscopic image and the adversarial loss function to obtain a trained microscopic image generation model.
2. The training method of the microscopic image generation model according to claim 1, characterized in that: The original microscopic image is processed by adding Gaussian noise to obtain a first microscopic image, which is specifically expressed as: X t = U(X0,t); Among them, X t represents the first microscopic image, X0 represents the original microscopic image, U(·) represents the Gaussian noise adding process, t represents the number of noise adding steps, t∈[1,2,…,T].
3. The training method of the microscopic image generation model according to claim 1, characterized in that: According to the first microscopic image, a second microscopic image is generated using a generator in a microscopic image generation model, which is specifically expressed as: in, represents the second microscopic image, X0 represents the original microscopic image, G(·) represents the generator, X t represents the first microscopic image, Z represents a random Gaussian variable, t represents the number of noise addition steps, t∈[1,2,…,T]; Gaussian noise is added to the second microscopic image to obtain a third microscopic image, which is specifically expressed as: in, represents the third microscopic image, U(·) represents the Gaussian noise adding process, and t-1 represents the number of noise adding steps applied at t-1.
4. The training method for a microscopic image generation model according to claim 1, characterized in that: The adversarial loss function is expressed as: in, represents the discriminator loss function, represents the generator loss function, Indicates the third microscopic image, X t-1 represents the microscopic image of the previous noise-added step of the first microscopic image, D(·) represents the discriminator, X t represents the first microscopic image, t represents the number of noise adding steps, t∈[1,2,…,T].
5. An application method of a microscopic image generation model, characterized in that: include: Determine a target image to be generated; Random Gaussian noise having the same shape as the target image matrix to be generated is input into the generator of a trained microscopic image generation model to generate a new high-quality microscopic image; the trained microscopic image generation model is a model trained according to the training method for a microscopic image generation model according to any one of claims 1 to 4.
6. A training system for a microscopic image generation model, characterized in that: include: An original microscopic image acquisition module, used for acquiring an original microscopic image; A first noise adding module, used for adding Gaussian noise to the original microscopic image to obtain a first microscopic image; A generation module, configured to generate a second microscopic image according to the first microscopic image by using a generator in a microscopic image generation model; the microscopic image generation model is a model composed of a diffusion model and a generative adversarial model, and the generative adversarial model includes a generator and a discriminator; A second noise adding module is used to add Gaussian noise to the second microscopic image to obtain a third microscopic image; the Gaussian noise added to the third microscopic image and the first microscopic image are both multi-step Gaussian noise, and the Gaussian noise added to the first microscopic image is t-step Gaussian noise, and the Gaussian noise added to the third microscopic image is t-1-step Gaussian noise; an identification module, configured to calculate, based on the third microscopic image and the first microscopic image, a probability that the third microscopic image is a reasonable denoised distribution of the first microscopic image using the discriminator in the microscopic image generation model; The model training module is used to train the microscopic image generation model according to the probability of the third microscopic image being a reasonable denoising distribution of the first microscopic image and the adversarial loss function to obtain a trained microscopic image generation model.
7. The training system for the microscopic image generation model according to claim 6, characterized in that: The first noise adding module adds Gaussian noise to the original microscopic image to obtain a first microscopic image, which is specifically expressed as: X t = U(X0,t); Among them, X t represents the first microscopic image, X0 represents the original microscopic image, U(·) represents the Gaussian noise adding process, t represents the number of noise adding steps, t∈[1,2,…,T].
8. The training system for the microscopic image generation model according to claim 6, characterized in that: The generation module generates a second microscopic image based on the first microscopic image using a generator in a microscopic image generation model, which is specifically expressed as: in, represents the second microscopic image, X0 represents the original microscopic image, G(·) represents the generator, X t represents the first microscopic image, Z represents a random Gaussian variable, t represents the number of noise addition steps, t∈[1,2,…,T]; The second noise adding module adds Gaussian noise to the second microscopic image to obtain a third microscopic image, which is specifically expressed as: in, represents the third microscopic image, U(·) represents the Gaussian noise adding process, and t-1 represents the number of noise adding steps applied at t-1.
9. The training system for the microscopic image generation model according to claim 6, characterized in that: The adversarial loss function is expressed as: in, represents the discriminator loss function, represents the generator loss function, Indicates the third microscopic image, X t-1 represents the microscopic image of the previous noise-added step of the first microscopic image, D(·) represents the discriminator, X t represents the first microscopic image, t represents the number of noise adding steps, t∈[1,2,…,T].
10. An application system for a microscopic image generation model, characterized in that: include: A target image determination module, used to determine a target image to be generated; A model application module is used to input random Gaussian noise of the same shape as the target image matrix to be generated into a trained microscopic image generation model to generate a new high-quality microscopic image; the trained microscopic image generation model is a model trained according to a microscopic image generation model training method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Wavelet denoising method for high-throughput gene sequencing image
CN105046665A
Translation of images of stained biological material
US20220058839A1
