Pathological section virtual staining image generation method based on diffusion model

Abstract

The invention discloses a pathological section virtual staining image generation method based on a diffusion model, and belongs to the technical field of medical image processing. The method comprises the following steps: firstly, passing through Hamp; e dyeing serves as a source image data set, and pathological section images subjected to IHC dyeing serve as a target image data set; preprocessing the source image, and converting the source image into a corresponding YUV color space; extracting a saliency map and a gradient map of the source image, and generating a weighted attention map; building a diffusion model by using a neural network, and inputting the preprocessed source image for fine tuning; repeatedly iterating until the loss function converges, and ending the training; and inputting a weighted attention map to perform reasoning of virtual dyeing. According to the method, weighted attention of a pathological section image is added in a reasoning part to serve as an image prompt input network, so that the cell structure of a finally generated image is controlled not to be changed, and meanwhile, color features of a target dyed image are reserved and emphasized; and the accuracy and the reconstruction effect of the generated virtual dyeing image are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for generating images of virtually stained pathological sections based on a diffusion model. Background Art

[0002] During the pathological diagnosis process, pathologists need to extract tissue samples from the lesion area through puncture or surgery, and perform a series of preparation operations such as fixation, dehydration, transparency, wax immersion, sectioning and staining for microscopic observation. These steps are time-consuming and require the use of certain harmful chemical reagents, which pose health risks and may affect the accuracy of diagnosis due to differences in technical level and quality of preparation materials. Although chemical staining methods improve the visibility of cell structures, they have problems such as cumbersome operation, high risk of reagent toxicity, unstable staining effects, and poor consistency of staining results between laboratories. For example, Harris et al. (2020) pointed out that the time cost and potential toxicity of traditional staining techniques pose a threat to the health of laboratory workers. 2021) emphasized that although existing staining techniques are effective in certain specific applications, their specificity and repeatability are still limited, which may affect the final diagnosis of pathologists.

[0003] In recent years, virtual staining has begun to replace traditional staining, avoiding the use of chemical reagents, improving safety and efficiency, and significantly enhancing staining consistency and flexibility. However, current virtual staining methods still face the problem of uneven performance when processing different types of tissue sections, resulting in the inability to effectively highlight certain specific pathological features. In recent years, diffusion models have achieved success in the field of grid transfer in imaging. By transferring the style features of different images to pathological images, a variety of virtual staining images can be generated, which effectively supports the diagnosis of complex cases and significantly saves staining time and costs. The application of this method provides important support for the diversified virtual staining of pathological sections. Summary of the invention

[0004] In view of the above technical problems, the present invention provides an image generation method for virtual staining of pathological sections based on a diffusion model, which belongs to the field of medical image processing technology. First, the H&E stained pathological section image is used as the source image data set, and the IHC stained pathological section image is used as the target image data set; the source image is preprocessed and converted to the corresponding YUV color space; the saliency map and gradient map of the source image are extracted to generate a weighted attention map; a diffusion model is built using a neural network, and the preprocessed source image is input for fine-tuning; it is repeatedly iterated until the loss function converges, and the training is terminated; the weighted attention map is input for virtual staining reasoning. The present invention adds the weighted attention of the pathological section image as an image prompt input network in the reasoning part to control the cell structure of the final generated image not to be changed, while retaining and emphasizing the color characteristics of the target stained image; improving the accuracy and reconstruction effect of the generated virtual stained image.

[0005] The specific steps of the technical solution of the present invention include:

[0006] S1, obtaining a data set: using a pathological section image stained with hematoxylin and eosin (H&E) as a source image data set; using a pathological section image stained with immunohistochemistry (IHC) as a target image data set;

[0007] S2, preprocessing the H&E stained images in the source image dataset to obtain the source images;

[0008] S3, generates a weighted attention map of the source image;

[0009] S4, constructing a virtual dyeing image generation network;

[0010] S5, training the virtual dye image generation network;

[0011] S6, reasoning using the virtual stained image generation network.

[0012] Further, in S2, the H&E staining image preprocessing process is as follows:

[0013] S2.1, format conversion: convert tiff format images to .png or .jpg format images;

[0014] S2.2, image cropping: crop the .png or .jpg format image into image blocks with a resolution of 512×512, ensuring that there is overlap in the x-axis and y-axis directions to avoid large color differences between adjacent image blocks in the generated image;

[0015] S2.3, image conversion: convert the cropped H&E stained image from RGB space to YUV color space according to the following formula, and then perform color statistics to better process color information; the image conversion selects ITU-R BT.601 (SDTV) conversion standard;

[0016]

[0017] Among them, Y, U, and V are the pixel values ​​in the color channels of the Gaussian filtered reference image; R, G, and B represent the red, green, and blue color channels of the H&E stained image, respectively;

[0018] S2.4, convert the colored YUV image back to RGB space according to the following formula:

[0019]

[0020] Further, in S3: the generation process of the weighted attention map is as follows:

[0021] S3.1, extracting high-dimensional features of H&E stained images through the Transformer model: first, the H&E stained images are divided into image blocks of 16×16 pixels; then, each image block is converted into a feature vector of length 768 through a linear projection layer; finally, each image block is added with a learnable positional encoding, so that the Transformer model can recognize the spatial positional relationship of each block;

[0022] S3.2, extracting image text features through a pre-trained language model: inputting the text prompt of the diagnosis result related to the H&E staining image into the pre-trained language model for encoding to obtain image text features corresponding to the high-dimensional features; the image text features include "cancer area", and the pre-trained language model is a bidirectional transformer model (Bidirectional Encoder Representations from Transformers, BERT) or a contrastive language-image pre-training model (Contrastive Language-Image Pre-Training, CLIP);

[0023] S3.3, obtaining weighted image block features through attention mechanism: by calculating the similarity between the image block and the image text feature, an attention weight is assigned to each image block to obtain the weighted image block feature, wherein the attention weight represents the relevance between the image block and the corresponding text prompt, and the similarity is the dot product attention;

[0024] S3.5, generate weighted attention map: The weighted image block features are summarized and visualized to obtain a weighted attention map, which helps doctors identify the areas that need attention in the image.

[0025] Further, in S4, the virtual dyeing image generation network construction process is as follows:

[0026] S4.1, build the basic network model:

[0027] First, an initial virtual image generation network is built based on the Stable Diffusion (SD) model: the SD model itself contains 16 traditional cross-attention layers, which fuse multimodal information through the cross-attention mechanism;

[0028] Then, an image cross-attention layer is inserted before each traditional cross-attention layer to enhance the image feature learning ability of the SD model; the image cross-attention layer is specifically used to process the interaction between image features and text features; the image cross-attention layer works in parallel with the traditional cross-attention layer to deeply fuse image features and text features through the self-attention mechanism;

[0029] S4.2: Image encoder optimization and feature extraction:

[0030] In the image feature extraction phase of the virtual stained image generation network, OpenCLIP ViT-H (Vision Transformer) is used as the image encoder; OpenCLIP ViT-H divides the input pathological H&E stained image into fixed-size image blocks, uses the self-attention mechanism to extract the high-dimensional features of each image block, and captures the spatial and contextual information in the image through multiple layers of Transformer layers; the OpenCLIP model optimizes the cross-modal learning ability and can effectively fuse image features with text prompts; the optimization and feature extraction of the image encoder provide input for the subsequent virtual stained image generation task;

[0031] S4.3: Embed a local control adapter ControlNet in each layer of the UNet of the Stable Diffusion model to introduce additional local control information;

[0032] Further, the local control adapter is designed as follows:

[0033] The input information of the local control adapter includes a grayscale image of a pathological slice and a weighted attention map generated in step S3, wherein the grayscale image of the pathological slice is the main input and the weighted attention map is the control condition; the control condition is processed using multi-scale convolution to extract features at different levels, wherein the multi-scale convolution includes FPN or lightweight ResNet; the features at different levels are from edge features to semantic features; the control condition is embedded into each layer of UNet as a diffusion process of an enhanced signal-guided diffusion model; residual connections are added to prevent the control signal from interfering with the original input features, thereby ensuring the stability of the generation process of the virtual image generation network.

[0034] Further, in S5, the training method of the virtual image generation network is as follows:

[0035] Using paired H&E and IHC stained images as input and target output, the virtual stained image generation network is trained. During the training process, the weighted attention map is used as auxiliary information to provide additional guidance signals through ControlNet.

[0036] Virtual staining of pathological H&E images should not only generate color results that match those of chemical reagent staining, but also maintain the structural information of cells and tissues. Therefore, the loss function L corresponding to the virtual image generation network includes the reconstruction loss L recon , structure retention loss L structure and the perceptual loss L perceptual , loss function L calculation formula:

[0037] L=L recon +L structure +L perceptual

[0038]

[0039] Among them, λ recon is the reconstruction loss weight, λ structure Keep the loss weight for the structure, λ perceptual is the perceptual loss weight, G(x) represents the generated virtual IHC stained image, y is the real IHC stained image, S(x) represents the structural feature map of the image, is the gradient operator of the image, φ l is the feature map of the layer of the pre-trained neural network;

[0040] Hyperparameters related to the virtual image generation network: The latent space dimension n of the variational autoencoder (VAE) in the Stable Diffusion model, the value of n is 64, 128 or 256, and n is preferably 128 or 256; the number of layers of the encoder and decoder of UNet is 4-6 layers; the dimension size of the ControlNet control signal determines the information richness of the control condition embedding, and the ControlNet dimension size is 64, 128 or 256; the reconstruction loss weight λ recon , value range [0.5, 1.0]; perceptual loss weight λ perceptual The value range is [0.5, 1.0]; the structure preservation loss weight λ structure Re-value range [0.5, 1.0];

[0041] Optimizer and learning rate related hyperparameters: learning rate, initial value is 2×10-4; optimizer, β1=0.9, β2=0.999 (AdamW default value); weight decay (Weight Decay) is 0.01 to prevent overfitting;

[0042] Hyperparameters related to training scheduling: the number of samples used in each training. Pathological images usually have high resolution, and the recommended number of samples is 4-8; training rounds (Epochs), initially set to 200, observe the performance of the validation set, and stop early if necessary.

[0043] Compared with the prior art, the present invention adopts the diffusion model as the main training framework to convert the H&E staining image into the IHC staining image in one step; the image quality and detail fidelity are improved; the diffusion model uses a step-by-step denoising method to generate images, making the generation process more refined and able to retain the complex details in the pathological slice image, which is particularly important for medical images, because the subtle differences in the diseased cells or tissue structures are the key to diagnosis; in contrast, GAN and CycleGAN are insufficient in generating details due to the mode collapse problem (mode collapse), which easily leads to the loss of some key pathological features. The stability of the generation process of the present invention and the gradual optimization characteristics of the diffusion model in the generation process ensure the stability of the generation results, and the training process is not prone to instability (such as inconsistent generation quality) like GAN, and the stable generation process makes the diffusion model more consistent in the virtual staining tasks of different pathological tissues. Compared with CycleGAN, the diffusion model of the present invention has a stronger cross-domain migration capability. CycleGAN often requires paired data for training, while the diffusion model can generate high-quality staining images under less paired data or unsupervised conditions, which is particularly beneficial for the situation where pathological data is insufficiently labeled. The details of the present invention are well maintained during the dyeing process, and the edge and texture information of the grayscale image is transmitted to the generated result through the multi-scale features of ControlNet, thereby avoiding the loss of details during the dyeing process. The present invention has enhanced flexibility, and the control conditions can be dynamically adjusted according to needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The present invention provides a flowchart of a method for generating an image of virtually stained pathological sections based on a diffusion model.

[0045] Figure 2 This is a structural diagram of the image generation method for virtually stained pathological sections based on a diffusion model provided by the present invention.

[0046] Figure 3 This is a structural diagram of a control adapter in the image generation method for virtually stained pathological sections based on a diffusion model provided by the present invention.

[0047] Figure 4 A schematic diagram of an image generation method for virtually stained pathological sections based on a diffusion model provided by the present invention. DETAILED DESCRIPTION

[0048] like Figure 1 , Figure 2 and Figure 4As shown, a method for generating images of virtually stained pathological sections based on a diffusion model belongs to the technical field of medical image processing. First, the H&E stained pathological section image is used as the source image data set, and the IHC stained pathological section image is used as the target image data set; the source image is preprocessed and converted to the corresponding YUV color space; the saliency map and gradient map of the source image are extracted to generate a weighted attention map; a diffusion model is built using a neural network, and the preprocessed source image is input for fine-tuning; it is repeatedly iterated until the loss function converges, and the training is terminated; the weighted attention map is input for virtual staining reasoning. The present invention adds the weighted attention of the pathological section image as an image prompt input network in the reasoning part to control the cell structure of the final generated image not to be changed, while retaining and emphasizing the color characteristics of the target stained image; and improves the accuracy and reconstruction effect of the generated virtual stained image.

[0049] The specific steps of the technical solution of the present invention include:

[0050] S1, obtaining a data set: using a pathological section image stained with hematoxylin and eosin (H&E) as a source image data set; using a pathological section image stained with immunohistochemistry (IHC) as a target image data set;

[0051] S2, preprocessing the H&E stained images in the source image dataset to obtain the source images;

[0052] S3, generates a weighted attention map of the source image;

[0053] S4, constructing a virtual dyeing image generation network;

[0054] S5, training the virtual dye image generation network;

[0055] S6, reasoning using the virtual stained image generation network.

[0056] In S2, the H&E staining image preprocessing process is as follows:

[0057] S2.1, format conversion: convert tiff format images to .png or .jpg format images;

[0058] S2.2, image cropping: crop the .png or .jpg format image into image blocks with a resolution of 512×512, ensuring that there is overlap in the x-axis and y-axis directions to avoid large color differences between adjacent image blocks in the generated image;

[0059] S2.3, image conversion: convert the cropped H&E stained image from RGB space to YUV color space according to the following formula, and then perform color statistics to better process color information; the image conversion selects ITU-R BT.601 (SDTV) conversion standard;

[0060]

[0061] Among them, Y, U, and V are the pixel values ​​in the color channels of the Gaussian filtered reference image; R, G, and B represent the red, green, and blue color channels of the H&E stained image, respectively;

[0062] S2.4, convert the colored YUV image back to RGB space according to the following formula:

[0063]

[0064] In S3: the generation process of the weighted attention map is as follows:

[0065] S3.1, extracting high-dimensional features of H&E stained images through the Transformer model: first, the H&E stained images are divided into image blocks of 16×16 pixels; then, each image block is converted into a feature vector of length 768 through a linear projection layer; finally, each image block is added with a learnable positional encoding, so that the Transformer model can recognize the spatial positional relationship of each block;

[0066] S3.2, extracting image text features through a pre-trained language model: inputting the text prompt of the diagnosis result related to the H&E staining image into the pre-trained language model for encoding to obtain image text features corresponding to the high-dimensional features; the image text features include "cancer area", and the pre-trained language model is a bidirectional transformer model (Bidirectional Encoder Representations from Transformers, BERT) or a contrastive language-image pre-training model (Contrastive Language-Image Pre-Training, CLIP);

[0067] S3.3, obtaining weighted image block features through attention mechanism: by calculating the similarity between the image block and the image text feature, an attention weight is assigned to each image block to obtain the weighted image block feature, wherein the attention weight represents the relevance between the image block and the corresponding text prompt, and the similarity is the dot product attention;

[0068] S3.5, generate weighted attention map: The weighted image block features are summarized and visualized to obtain a weighted attention map, which helps doctors identify the areas that need attention in the image.

[0069] In S4, the virtual dyeing image generation network construction process is as follows:

[0070] S4.1, build the basic network model:

[0071] First, an initial virtual image generation network is built based on the Stable Diffusion (SD) model: the SD model itself contains 16 traditional cross-attention layers, which fuse multimodal information through the cross-attention mechanism;

[0072] Then, an image cross-attention layer is inserted before each traditional cross-attention layer to enhance the image feature learning ability of the SD model; the image cross-attention layer is specifically used to process the interaction between image features and text features; the image cross-attention layer works in parallel with the traditional cross-attention layer to deeply fuse image features and text features through the self-attention mechanism;

[0073] S4.2: Image encoder optimization and feature extraction:

[0074] In the image feature extraction phase of the virtual stained image generation network, OpenCLIP ViT-H (Vision Transformer) is used as the image encoder; OpenCLIP ViT-H divides the input pathological H&E stained image into fixed-size image blocks, uses the self-attention mechanism to extract the high-dimensional features of each image block, and captures the spatial and contextual information in the image through multiple layers of Transformer layers; the OpenCLIP model optimizes the cross-modal learning ability and can effectively fuse image features with text prompts; the optimization and feature extraction of the image encoder provide input for the subsequent virtual stained image generation task;

[0075] S4.3: Embed a local control adapter ControlNet in each layer of the UNet of the Stable Diffusion model to introduce additional local control information;

[0076] like Figure 3 As shown, the local control adapter is designed as follows:

[0077] The input information of the local control adapter includes a grayscale image of a pathological slice and a weighted attention map generated in step S3, wherein the grayscale image of the pathological slice is the main input and the weighted attention map is the control condition; the control condition is processed using multi-scale convolution to extract features at different levels, wherein the multi-scale convolution includes FPN or lightweight ResNet; the features at different levels are from edge features to semantic features; the control condition is embedded into each layer of UNet as a diffusion process of an enhanced signal-guided diffusion model; residual connections are added to prevent the control signal from interfering with the original input features, thereby ensuring the stability of the generation process of the virtual image generation network.

[0078] Further, in S5, the training method of the virtual image generation network is as follows:

[0079] Using paired H&E and IHC stained images as input and target output, the virtual stained image generation network is trained. During the training process, the weighted attention map is used as auxiliary information to provide additional guidance signals through ControlNet.

[0080] Virtual staining of pathological H&E images should not only generate color results that match those of chemical reagent staining, but also maintain the structural information of cells and tissues. Therefore, the loss function L corresponding to the virtual image generation network includes the reconstruction loss L recon , structure retention loss L structure and the perceptual loss L perceptual , loss function L calculation formula:

[0081] L=L recon +L structure +L perceptual

[0082]

[0083] Among them, λ recon is the reconstruction loss weight, λ structure Keep the loss weight for the structure, λ perceptual is the perceptual loss weight, G(x) represents the generated virtual IHC stained image, y is the real IHC stained image, S(x) represents the structural feature map of the image, is the gradient operator of the image, φ l is the feature map of the layer of the pre-trained neural network;

[0084] Hyperparameters related to the virtual image generation network: The latent space dimension n of the variational autoencoder (VAE) in the Stable Diffusion model, the value of n is 64, 128 or 256, and n is preferably 128 or 256; the number of layers of the encoder and decoder of UNet is 4-6 layers; the dimension size of the ControlNet control signal determines the information richness of the control condition embedding, and the ControlNet dimension size is 64, 128 or 256; the reconstruction loss weight λ recon , value range [0.5, 1.0]; perceptual loss weight λ perceptual The value range is [0.5, 1.0]; the structure preservation loss weight λ structure Re-value range [0.5, 1.0];

[0085] Optimizer and learning rate related hyperparameters: learning rate, initial value is 2×10-4; optimizer, β1=0.9, β2=0.999 (AdamW default value); weight decay (Weight Decay) is 0.01 to prevent overfitting;

[0086] Hyperparameters related to training scheduling: the number of samples used in each training. Pathological images usually have high resolution, and the recommended number of samples is 4-8; training rounds (Epochs), initially set to 200, observe the performance of the validation set, and stop early if necessary.

Claims

1. A method for generating images of virtually stained pathological sections based on a diffusion model, characterized in that: The virtual dyeing image generation method comprises the following steps: S1, obtaining a data set: using the pathological section images stained with H&E as the source image data set; using the pathological section images stained with IHC as the target image data set; S2, preprocessing the H&E stained images in the source image dataset to obtain the source images; S3, generates a weighted attention map of the source image; S4, constructing a virtual dyeing image generation network; S5, training the virtual dye image generation network; S6, reasoning using the virtual stained image generation network.

2. The method for generating an image of a pathological section virtually stained based on a diffusion model according to claim 1, characterized in that: In S2, the preprocessing process of the H&E staining image is as follows: S2.1, format conversion: convert tiff format images to .png or .jpg format images; S2.2, image cropping: crop the .png or .jpg format image into image blocks with a resolution of 512×512; S2.3, image conversion: convert the cropped H&E stained image from RGB space to YUV color space according to the following formula, and then perform color statistics; the conversion method selects ITU-R BT.601 conversion standard; Among them, Y, U, and V are the pixel values ​​in the color channels of the Gaussian filtered reference image; R, G, and B represent the red, green, and blue color channels of the H&E stained image, respectively; S2.4, convert the YUV image after color conversion operation back to RGB space, the conversion formula is as follows:

3. The method for generating an image of a pathological section virtual staining based on a diffusion model according to claim 1, characterized in that: In S3, the generation process of the weighted attention map is as follows: S3.1, extracting high-dimensional features of H&E stained images through the Transformer model: first, the H&E stained images are divided into image blocks of 16×16 pixels; then, each image block is converted into a feature vector of length 768 through a linear projection layer; finally, a learnable position encoding is added to each image block, so that the Transformer model can recognize the spatial position relationship of each block; S3.2, extracting image text features through a pre-trained language model: inputting the text prompt of the diagnosis result related to the H&E staining image into the pre-trained language model for encoding to obtain image text features corresponding to the high-dimensional features; the image text features include "cancer area", and the pre-trained language model is a bidirectional transformer model or a contrastive language-image pre-trained model; S3.3, obtaining weighted image block features through attention mechanism: calculating the similarity between image blocks and image text features, assigning an attention weight to each image block, and obtaining weighted image block features, wherein the attention weight represents the relevance between the image block and the corresponding text prompt, and the similarity is dot product attention; S3.5, generate weighted attention map: the weighted image block features are summarized and visualized to obtain the weighted attention map.

4. The method for generating an image of a pathological section virtual staining based on a diffusion model according to claim 1, characterized in that: In S4, the virtual dyeing image generation network construction process is as follows: S4.1, build the basic network model: First, the initial virtual image generation network is built based on the SD model: the SD model itself contains 16 traditional cross-attention layers, which fuse multimodal information through the cross-attention mechanism; Then, an image cross-attention layer is inserted before each traditional cross-attention layer to enhance the image feature learning ability of the SD model; the image cross-attention layer works in parallel with the traditional cross-attention layer to deeply fuse image features and text features through the self-attention mechanism; S4.2: Image encoder optimization and feature extraction: In the image feature extraction phase of the virtual dyeing image generation network, OpenCLIPViT-H is used as the image encoder; OpenCLIPViT-H divides the input pathological H&E stained images into fixed-size image blocks, uses the self-attention mechanism to extract high-dimensional features of each image block, and captures the spatial and contextual information within the image through multiple layers of Transformer layers; the OpenCLIP model optimizes cross-modal learning capabilities and fuses image features with text prompts; S4.3: A local control adapter ControlNet is embedded in each layer of the UNet of the Stable Diffusion model to introduce additional local control information.

5. The method for generating an image of a pathological section virtual staining based on a diffusion model according to claim 4, characterized in that: The local control adapter is designed as follows: The input information of the local control adapter includes a grayscale image of a pathological slice and a weighted attention map generated in step S3, wherein the grayscale image of the pathological slice is the main input and the weighted attention map is the control condition; the control condition is processed using multi-scale convolution to extract features at different levels; the control condition is embedded into each layer of UNet as a diffusion process of an enhanced signal-guided diffusion model; residual connections are added to prevent the control signal from interfering with the original input features, thereby ensuring the stability of the virtual image generation network generation process; the multi-scale convolution includes FPN or lightweight ResNet; the features at different levels are from edge features to semantic features.

6. The method for generating an image of a pathological section virtual staining based on a diffusion model according to claim 1, characterized in that: In S5, the training method of the virtual image generation network is as follows: Using paired H&E and IHC stained images as input and target output, the virtual stained image generation network is trained. During the training process, the weighted attention map is used as auxiliary information to provide additional guidance signals through ControlNet. Virtual staining of pathological H&E images should not only generate color results that match those of chemical reagent staining, but also maintain the structural information of cells and tissues. Therefore, the loss function L corresponding to the virtual image generation network includes the reconstruction loss L recon , structure retention loss L structure and the perceptual loss L perceptual , loss function L calculation formula: L=L recon +L structure +L perceptual Among them, λ recon is the reconstruction loss weight, λ structure Keep the loss weight for the structure, λ perceptual is the perceptual loss weight, G(x) represents the generated virtual IHC stained image, y is the real IHC stained image, S(x) represents the structural feature map of the image, is the gradient operator of the image, φ l is the feature map of the layer of the pre-trained neural network; Hyperparameters related to the virtual image generation network: The latent space dimension n of the variational autoencoder in the Stable Diffusion model, the value of n is 64, 128 or 256, and n is preferably 128 or 256; the number of layers of the encoder and decoder of UNet is 4-6 layers; the dimension size of the ControlNet control signal, the ControlNet dimension size is 64, 128 or 256; the reconstruction loss weight λ recon , value range [0.5, 1.0]; perceptual loss weight λ perceptual The value range is [0.5, 1.0]; the structure preservation loss weight λ structure Re-value range [0.5, 1.0]; Hyperparameters related to optimizer and learning rate: the initial value of learning rate is 2×10-4; β1=0.9, β2=0.999 of optimizer; weight decay is 0.01 to prevent overfitting; Hyperparameters related to training scheduling: the number of samples used in each training. Pathological images usually have high resolution, and the recommended number of samples is 4-8; the number of training rounds, initially set to 200, observe the performance of the validation set, and stop early if necessary.

7. An electronic device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the method according to any one of claims 1 to 6.

Images