A method for generating magnified endoscopic images based on diffusion model guided by pathological information

Abstract

The present invention provides a method for generating magnified endoscopic images based on a diffusion model guided by pathological information, including obtaining a real image of a magnified endoscope, a mask image and a text prompt of pathological information, and constructing a training and test data set; constructing a magnified endoscopic image generation model based on pathological information guidance and training it through a training data set to obtain a trained magnified endoscopic image generation model; using a test data set for testing to evaluate its performance on unknown data, and the one that is qualified is the final magnified endoscopic image generation model; applying the final magnified endoscopic image generation model to the magnified endoscopic image generation work to obtain the magnified endoscopic image generated by the model. This method is suitable for the work of generating magnified endoscopic images, and uses the method of guiding pathological information and background mask images to accurately generate magnified endoscopic images of different pathologies, effectively solving the data imbalance problem of downstream segmentation and classification tasks of magnified endoscopic images.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image generation, and in particular to a method for generating a diffusion model magnified endoscopic image based on pathological information guidance. Background Art

[0002] In the field of medical image analysis, the success of deep learning technology depends on large-scale, diverse and high-quality data, which can effectively improve the generalization ability of the model and reduce the risk of overfitting. However, high-quality annotated data in the medical field is often scarce, especially in image generation tasks related to pathological information. This lack of data has become a major bottleneck in the development of robust and efficient deep learning models. Therefore, how to generate realistic and diverse medical images has become an important research direction in academia and industry.

[0003] In recent years, diffusion models, as a powerful generative model, have shown excellent performance in the field of image generation, especially in generating high-quality images with complex structures and rich details. Diffusion models can reconstruct highly realistic images from random noise by gradually denoising the image. Therefore, such models are widely used in various image synthesis tasks. However, traditional diffusion models lack the ability to control specific medical information (such as pathological information), which is crucial for generating medical images that meet clinical needs.

[0004] In view of this, this application is filed. Summary of the invention

[0005] The present invention provides a method for generating a diffusion model magnified endoscopic image based on pathological information guidance, which can at least partially improve the above-mentioned problem.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for generating a diffusion model magnified endoscopic image based on pathological information guidance, comprising:

[0008] Acquire a real image of the current magnifying endoscope, a mask image corresponding to the real image, and a text prompt of pathological information, and construct a training data set and a test data set according to the real image, the mask image, and the text prompt of pathological information, wherein the mask image is related to the background information;

[0009] Constructing a magnifying endoscopic image generation model guided by pathological information, and using the training data set to train the magnifying endoscopic image generation model guided by pathological information to obtain a trained magnifying endoscopic image generation model;

[0010] Using the test data set to test the trained magnifying endoscopic image generation model, evaluating the performance of the trained magnifying endoscopic image generation model on unknown data, and when the evaluation result is qualified, obtaining the final magnifying endoscopic image generation model;

[0011] The current image to be generated is acquired, and the image to be generated is input into the final magnifying endoscopic image generation model to obtain a single or batch diffusion model magnified endoscopic image.

[0012] In summary, the specific steps of the method for generating magnified endoscopic images based on the diffusion model guided by pathological information include: obtaining a real image of the magnifying endoscope, its corresponding Mask image about the background information and its corresponding text prompt of the pathological information, and constructing a training data set and a test data set; constructing a magnifying endoscopic image generation model based on the guidance of pathological information and training it through the training data set to obtain a trained magnifying endoscopic image generation model; using the test data set to test the trained magnifying endoscopic image generation model to evaluate its performance on unknown data, and the one that passes the evaluation is the final magnifying endoscopic image generation model; applying the final magnifying endoscopic image generation model to the magnifying endoscopic image generation work to obtain the magnified endoscopic image generated by the model.

[0013] In short, the method for generating magnified endoscopic images based on a diffusion model guided by pathological information combines pathological information with a diffusion model, and uses different pathological categories of magnified endoscopic images as generation conditions to ensure that the generated images are not only diverse, but also retain the key structural features in medical images. In particular, this method introduces geometric information based on background masks, so that the generated magnified endoscopic images can retain the original structure while being more in line with the actual pathological manifestations. Through this mechanism guided by pathological information, the diffusion model can more effectively generate magnified endoscopic images that meet clinical needs, thereby alleviating the challenges caused by the scarcity of data in certain pathological categories. At the same time, this method provides a balanced and rich dataset for subsequent segmentation and classification tasks of magnified endoscopic images, significantly improving the accuracy and robustness of downstream tasks.

[0014] Compared with the prior art, the advantages of the method for generating magnified endoscopic images based on the diffusion model guided by pathological information are as follows: 1. By combining pathological information and background mask as generation conditions, the diffusion model can generate magnified endoscopic images that meet different pathological categories, ensuring that the generated images are both diverse and have high medical relevance and structural accuracy, which helps to improve the accuracy in clinical applications. 2. It effectively alleviates the data scarcity problem of magnified endoscopic images in specific pathological categories, can generate diverse and realistic images, and provides a more balanced training data set, helping to improve the model performance of downstream tasks such as image classification and segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of a method for generating a diffusion model magnified endoscopic image based on pathological information guidance provided by an embodiment of the present invention;

[0016] Figure 2 It is a flowchart for implementing the training of a diffusion model magnified endoscopic image generation model guided by pathological information provided by an embodiment of the present invention;

[0017] Figure 3 This is a magnifying endoscope generation effect diagram on a test data set provided by an embodiment of the present invention, and is displayed using a graphical user interface created with the Gradio library. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] refer to Figure 1 As shown, the first embodiment of the present invention discloses a method for generating a diffusion model magnified endoscopic image based on pathological information guidance, which can be executed by a diffusion model magnified endoscopic image generating device based on pathological information guidance (hereinafter referred to as a generating device), and in particular, executed by one or more processors in the generating device to implement the following method:

[0020] S1, obtaining a real image of the current magnifying endoscope, a mask image corresponding to the real image, and a text prompt of pathological information, and constructing a training data set and a test data set according to the real image, the mask image, and the text prompt of pathological information, wherein the mask image is related to background information;

[0021] Specifically, step S1 includes: obtaining a magnified endoscopic image, magnified endoscopic background information, and magnified endoscopic pathological information of the current patient, taking the magnified endoscopic image as a real image, taking the magnified endoscopic pathological information as a corresponding pathological information text prompt, and generating a Mask image according to the magnified endoscopic background information as a corresponding mask image;

[0022] The real image, the mask image and the text prompt of the pathological information are divided to construct a training data set and a test data set.

[0023] Preferably, the pathological information of the magnifying endoscope includes four categories, namely, hyperplastic polyps, adenomas, sessile serrated lesions and cancers.

[0024] In this embodiment, the real image of the magnifying endoscope, the corresponding mask image of the background information and the corresponding text prompt of the pathological information are obtained to construct a training data set and a test data set. Specifically, the magnifying endoscope image of the patient provided by the hospital is used as the real image, the mask generated by the background information of the magnifying endoscope is used as the corresponding mask image, and the pathological information about the magnifying endoscope provided by the doctor is used as the corresponding text prompt information. The obtained magnifying endoscope image is processed, and then the training data set and the test data set are constructed. Among them, the pathological information of the magnifying endoscope image is divided into four categories, namely, hyperplastic polyps, adenomas, sessile serrated lesions and cancer.

[0025] See also Figure 2 , Figure 2 This is the architecture of the diffusion model magnified endoscopic image generation model based on pathological information guidance and the principle diagram of its training in this embodiment, showing that the model uses pathological information text prompts and background masks as auxiliary information to train and generate magnified endoscopic images. Among them, Figure 2 The English text in the pathological information prompt box is “Cellular magnifying endoscopy of XXX, XXX: 0: hyperplastic polyps, 1: adenoma cells, 2: sessile serrated lesions, 3: malignant tumors”; the Chinese explanation is: Cellular magnifying endoscopy of XXX lesions, where 0 represents hyperplastic polyps, 1 represents adenoma, 2 represents sessile serrated lesions, and 3 represents invasive carcinoma.

[0026] S2, constructing a magnifying endoscopic image generation model guided by pathological information, and using the training data set to train the magnifying endoscopic image generation model guided by pathological information to obtain a trained magnifying endoscopic image generation model;

[0027] Specifically, step S2 includes: constructing a magnifying endoscopic image generation model guided by pathological information, and performing data preprocessing on the training data set;

[0028] The training data set after data preprocessing is used to train the magnifying endoscopic image generation model guided by pathological information to obtain a trained magnifying endoscopic image generation model.

[0029] Preferably, data preprocessing is performed on the training data set, specifically:

[0030] Performing loading initialization processing on the training data set, and setting the data loading path, data processing method and balanced sampling method;

[0031] Read each preset sample source image and target image, perform RGB color space conversion and normalization processing, normalize the pixel value of the source image to the range of [0,1], and normalize the pixel value of the target image to the range of [-1,1], wherein the preset sample includes the source image file name, the target image file name and the pathological information text prompt;

[0032] According to the text prompt of each preset sample, the sampling weight is calculated, wherein the sampling weight is inversely proportional to the frequency of the text prompt in the data set;

[0033] The processed image data, text prompt information and corresponding file names are packaged into a dictionary structure and returned for model training.

[0034] Preferably, the training data set after data preprocessing is used to train the magnifying endoscopic image generation model guided by pathological information to obtain a trained magnifying endoscopic image generation model, specifically:

[0035] The training data set after data preprocessing is used as input data and input into a magnifying endoscopic image generation model guided by pathological information;

[0036] Input the preprocessed pathological information text prompts in the training data set into the OpenCLIP text encoder of the pathological information-guided magnified endoscopic image generation model, and perform text tokenization, embedding, position encoding and Transformer layer processing in sequence to obtain a high-dimensional semantic embedding vector context;

[0037] The source image and semantic embedding vector context in the training data set are input into the ControlNet network, the time step embedding is calculated, the Mask image is used as the hint feature, the convolution layer and SiLU processing are performed, and the preset auxiliary information and hint features are combined to obtain the feature guided_hint;

[0038] Extract the main feature map h from the real image, combine the auxiliary information and the main feature map h for further feature extraction and fusion to generate a feature map h0, perform element-wise addition on the feature guided_hint and the feature map h0, directly integrate the geometric information of the training data set into the feature map h0 to generate a feature map h1, and perform enhancement and smoothing on the feature map h1 to obtain a feature map h2;

[0039] The real image is input into the encoder, and a latent space representation is generated according to the encoder method. The feature map is converted into a latent space representation for sampling the latent vector z through the quant_conv convolution layer. During the training process of the stable diffusion model, noise is gradually added to the latent vector z at multiple time steps t to generate a noise image x_noisy, and the formula is: ,in, is the noise image corresponding to time step t , is the noise scaling factor corresponding to time step t, is random noise sampled from a standard normal distribution;

[0040] The feature map h2, the noise image x_noisy, the time step t and the semantic embedding vector context are combined to form complete conditional information, which is then transmitted to the Stable Diffusion drawing tool to calculate and predict the noise of the current step and apply it to the denoising process. The expression of the denoising process is: ,in, is the noise image corresponding to time step t-1 , is the noise that the model predicts at the current step, is the preset input condition;

[0041] Use the loss function to measure the noise of the model's prediction at the current step The actual noise added The gap between them, the expression of the loss function is: ,in, For the time step t, the original image ,noise Expected value;

[0042] Minimize the noise in the model's predictions for the current step The actual noise added When the gap reaches a preset value, the training is terminated to obtain a trained magnified endoscopic image generation model.

[0043] In this embodiment, a magnifying endoscopic image generation model guided by pathological information is constructed and trained by a training data set to obtain a trained magnifying endoscopic image generation model. Specifically, a magnifying endoscopic image generation model guided by pathological information is constructed; data of the training data set is loaded and data preprocessing is performed; the constructed magnifying endoscopic image generation model is trained by the data after data preprocessing to obtain a trained magnifying endoscopic image generation model.

[0044] Specifically, in this embodiment, the specific steps of loading the data of the training data set and performing data preprocessing are: initializing the loading of the training data set data, setting the data loading path, data processing method, balanced sampling method, etc. First, read a JSON file containing multiple samples, each sample including a source image file name, a target image file name, and a text prompt of pathological information. The data of each sample is stored in a data list, and the source image and target image of each sample are read, RGB color space converted and normalized, wherein the pixel value of the source image is normalized to the range of [0,1] to standardize the input data, and the pixel value of the target image is normalized to the range of [-1,1] to meet the training requirements of the generated model. Then, the sampling weight is calculated according to the text prompt of each sample, and the sampling weight is inversely proportional to the frequency of the text prompt in the data set, so as to balance the category distribution in the data set. Finally, the processed image data, text prompt information and corresponding file name are packaged into a dictionary structure and returned for model training.

[0045] In this embodiment, the constructed magnifying endoscopic image generation model is trained by the data after data preprocessing, and the specific steps of obtaining the trained magnifying endoscopic image generation model are as follows: the processed source image, target image and pathological information text prompt are loaded from the training data set and input into the magnifying endoscopic image generation model. The pathological information text prompt is input into the OpenCLIP text encoder, and the text is first tokenized. Specifically, the text prompt is tokenized by a function to generate a 'tokens' tensor, and each token in the tokenized 'tokens' corresponds to a word. Secondly, embedding and position encoding are performed, specifically, 'tokens' is input into the embedding layer of the model to generate an embedding representation x, with a shape of [n_ctx, d_model], where n_ctx is the context length and d_model is the embedding dimension. The embedded representation x is added to the position embedding to introduce its position information in the sequence for each token. Finally, the Transformer layer is processed. Specifically, the embedded representation x will be used as the input of the Transformer layer. After being processed by multiple attention mechanisms and feedforward networks, the processed features are intercepted in the penultimate layer to output a high-dimensional semantic embedding vector (context).

[0046] The source image, that is, the background mask and context of the magnified endoscopic image, are input into ControlNet. First, the time step embedding is calculated. Specifically, ControlNet receives the current time step (timesteps) information and encodes the timesteps into a time embedding vector (t_emb). t_emb then passes through two linear layers and an activation function (SiLU) to generate the final time embedding (emb). Secondly, the geometric information is processed. Specifically, the background mask is input as a hint into the input_hint_block module. After a series of convolutional layers and SiLU processing, the geometric features are gradually extracted as higher-level representations. In this process, the feature map gradually reduces the spatial resolution through downsampling, while increasing the number of channels to better combine with subsequent processing stages. The use of a zero convolution layer in the last layer of the module ensures that the feature map has zero impact on the network at the beginning of training, which helps the model gradually learn how to use geometric information during training. In addition, emb and context are combined with the hint feature as auxiliary information to generate guided_hint. The second is the fusion of features. Specifically, the main feature map h is first extracted from the target image, that is, the real image of the magnifying endoscope, and h is combined with emb and context for further feature extraction and fusion to generate a richer feature map h0. Finally, guided_hint and h0 are added at an element level to directly integrate the geometric information into the current h0 to generate feature map h1. Finally, h1 is processed, specifically h1 is further processed by the middle_block module to obtain feature map h2. This module includes modules such as residual blocks and attention blocks (or spatial transformers), which are responsible for strengthening the dependencies between features, retaining and strengthening temporal and spatial features, and finally through the smoothing operation of the zero convolution layer, ensuring that the impact on the output in the early stage of training is minimized, thereby allowing the model to gradually learn how to effectively use these feature information during the training process.

[0047] Input the real image of the magnifying endoscope into the encoder (Autoencoder). First, the image will generate a latent space representation through the encoder method. During this process, the encoder generates feature maps for the mean and standard deviation, and then converts these feature maps into a latent space representation for sampling the latent vector z through the quant_conv convolution layer. During the training process of the stable diffusion model (Stable Diffusion), noise will be gradually added to z at multiple time steps t. This step is to simulate the process of an image gradually changing from high quality to complete noise. Specifically, given a time step t, the model will add a time-step-related noise amount to z to generate a noisy image x_noisy. This process can be expressed by the formula: ,in, is the noise scaling factor corresponding to time step t, It is random noise sampled from a standard normal distribution. In the next step, h2 is combined with the current noise image x_noisy, time step t, and context to form a complete conditional information passed to Stable Diffusion. During the forward propagation of Stable Diffusion, more noise will be gradually added to the image as the time step advances. During the back propagation process, the model will calculate and predict the noise of the current step based on the current noise image x_noisy, time step t, context, and h2 features passed from ControlNet. The denoising process can be expressed as: ,in, is the noise of the model prediction at the current step, Represents the preset input conditions, that is, the input of various preset conditions, where the various preset conditions include background mask features, pathological information text features, and real image features. The loss function in the denoising process can be expressed as: ,in, Represents the original image at time step t ,noise The expected value of the model is measured by the loss function. The actual noise added By minimizing this gap, the model gradually learns how to effectively remove noise, thereby effectively updating the latent space representation of the image. Finally, the latent space representation z after the denoising step is decoded into the final image through the decoder.

[0048] S3, using the test data set to test the trained magnifying endoscopic image generation model, evaluating the performance of the trained magnifying endoscopic image generation model on unknown data, and when the evaluation result is qualified, obtaining the final magnifying endoscopic image generation model;

[0049] Specifically, step S3 includes: inputting the background mask image and the corresponding pathological information text prompt in the test data set into the trained magnified endoscopic image generation model to obtain a newly generated magnified endoscopic image;

[0050] The newly generated magnified endoscopic image is evaluated, and the newly generated magnified endoscopic image is compared with the real image to generate an evaluation result. When the evaluation result reaches a preset result value, the evaluation is qualified, and a final magnified endoscopic image generation model is obtained.

[0051] In this embodiment, the trained magnifying endoscopic image generation model is tested using a test data set to evaluate its performance on unknown data, and the one that passes the evaluation is the final magnifying endoscopic image generation model.

[0052] Specifically, the background mask image and its corresponding pathological information text prompt in the test data set are input into the trained magnified endoscopic image generation model to obtain a newly generated magnified endoscopic image. The newly generated image of the model is compared with the corresponding real image, and the performance of the model on the test data set is evaluated. The one that passes the evaluation is the final magnified endoscopic image generation model.

[0053] S4, obtaining the current image to be generated, inputting the image to be generated into the final magnifying endoscopic image generation model, and obtaining a single or batch diffusion model magnified endoscopic image.

[0054] In this embodiment, the magnifying endoscopic image generation model finally obtained is applied to the magnifying endoscopic image generation work to obtain the magnifying endoscopic image generated by the model. A single magnifying endoscopic image can be generated according to the trained model; specifically, the open source Gradio library is used to create a graphical user interface, allowing the user to easily interact with the model through the web interface, so that the user can conveniently upload the background Mask image of the magnifying endoscope, enter the text prompt of the pathological information, adjust the parameters, and finally click to generate a magnifying endoscopic image guided by the background Mask and text prompt. It is also possible to generate batches of magnifying endoscopic images according to the trained model. Specifically, on the server side, all background Mask images need to be stored in a specified directory first. Then load the trained model path and configure the pathological information text prompt corresponding to the background Mask. After all paths and parameters are set, run the generate_synthetic_copy() function, which will traverse all background Mask images in the specified directory, generate multiple corresponding magnifying endoscopic images for each background Mask, and save the generated images in the specified output directory.

[0055] See also Figure 3 , Figure 3 The effect of generating a magnified endoscope on the test data set by the method in this embodiment is shown. In the graphical user interface created by the Gradio library, the pathological information text prompt word and background mask are used as input, and the corresponding magnified endoscope image can be obtained after adjusting the parameters and clicking Generate. Among them, Figure 3The English name is “Cellularmagnifying endoscopy for pedunculated serrated lesions of polyps”, and the Chinese explanation is: Cellularmagnifying endoscopy for pedunculated serrated lesions of polyps.

[0056] In summary, the method for generating magnified endoscopic images based on a diffusion model guided by pathological information combines pathological information with a diffusion model, and uses different pathological categories of magnified endoscopic images as generation conditions, thereby ensuring that the generated images are not only diverse, but also retain the key structural features in medical images. In particular, this method introduces geometric information based on background masks, so that the generated magnified endoscopic images can retain the original structure while being more in line with the actual pathological manifestations. In short, the method for generating magnified endoscopic images based on a diffusion model guided by pathological information is suitable for the generation of magnified endoscopic images. It uses pathological information and background mask images to accurately generate magnified endoscopic images of different pathologies, effectively solving the data imbalance problem of downstream segmentation and classification tasks of magnified endoscopic images.

[0057] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for generating magnified endoscopic images based on a diffusion model guided by pathological information, characterized in that: include: Acquire a real image of the current magnifying endoscope, a mask image corresponding to the real image, and a text prompt of pathological information, and construct a training data set and a test data set according to the real image, the mask image, and the text prompt of pathological information, wherein the mask image is related to the background information; Constructing a magnifying endoscopic image generation model guided by pathological information, and using the training data set to train the magnifying endoscopic image generation model guided by pathological information to obtain a trained magnifying endoscopic image generation model; Using the test data set to test the trained magnifying endoscopic image generation model, evaluating the performance of the trained magnifying endoscopic image generation model on unknown data, and when the evaluation result is qualified, obtaining the final magnifying endoscopic image generation model; Acquire the current image to be generated, input the image to be generated into the final magnified endoscopic image generation model, and obtain a single or batch diffusion model magnified endoscopic image; The real image of the current magnifying endoscope, the mask image corresponding to the real image, and the text prompt of the pathological information are obtained, and the training data set and the test data set are constructed according to the real image, the mask image, and the text prompt of the pathological information. Specifically, Obtaining a magnified endoscopic image, magnified endoscopic background information, and magnified endoscopic pathological information of the current patient, taking the magnified endoscopic image as a real image, taking the magnified endoscopic pathological information as a corresponding pathological information text prompt, and generating a Mask image according to the magnified endoscopic background information as a corresponding mask image; The real image, the mask image and the pathological information text prompt are divided to construct a training data set and a test data set; Constructing a magnifying endoscopic image generation model guided by pathological information, using the training data set to train the magnifying endoscopic image generation model guided by pathological information, and obtaining a trained magnifying endoscopic image generation model, specifically: Constructing a magnifying endoscopic image generation model guided by pathological information, and performing data preprocessing on the training data set; Using the training data set after data preprocessing to train the magnifying endoscopic image generation model guided by pathological information, to obtain a trained magnifying endoscopic image generation model; The training data set after data preprocessing is used to train the magnifying endoscopic image generation model guided by pathological information to obtain a trained magnifying endoscopic image generation model, specifically: The training data set after data preprocessing is used as input data and input into a magnifying endoscopic image generation model guided by pathological information; Input the preprocessed pathological information text prompts in the training data set into the OpenCLIP text encoder of the pathological information-guided magnified endoscopic image generation model, and perform text tokenization, embedding, position encoding and Transformer layer processing in sequence to obtain a high-dimensional semantic embedding vector context; The source image and semantic embedding vector context in the training data set are input into the ControlNet network, the time step embedding is calculated, the Mask image is used as the hint feature, the convolution layer and SiLU processing are performed, and the preset auxiliary information and hint features are combined to obtain the feature guided_hint; Extract the main feature map h from the real image, combine the auxiliary information and the main feature map h for further feature extraction and fusion to generate a feature map h0, perform element-wise addition on the feature guided_hint and the feature map h0, directly integrate the geometric information of the training data set into the feature map h0 to generate a feature map h1, and perform enhancement and smoothing on the feature map h1 to obtain a feature map h2; The real image is input into the encoder, and a latent space representation is generated according to the encoder method. The feature map is converted into a latent space representation for sampling the latent vector z through the quant_conv convolution layer. During the training process of the stable diffusion model, noise is gradually added to the latent vector z at multiple time steps t to generate a noise image x_noisy, and the formula is: ,in, is the noise image corresponding to time step t , is the noise scaling factor corresponding to time step t, is random noise sampled from a standard normal distribution; The feature map h2, the noise image x_noisy, the time step t and the semantic embedding vector context are combined to form complete conditional information, which is then transmitted to the Stable Diffusion drawing tool to calculate and predict the noise of the current step and apply it to the denoising process. The expression of the denoising process is: ,in, is the noise image corresponding to time step t-1 , is the noise that the model predicts at the current step, is the preset input condition; Use the loss function to measure the noise of the model's prediction at the current step The actual noise added The gap between them, the expression of the loss function is: ,in, For the time step t, the original image ,noise Expected value; Minimize the noise in the model's predictions for the current step The actual noise added When the gap reaches a preset value, the training is terminated to obtain a trained magnified endoscopic image generation model.

2. The method for generating a diffusion model magnified endoscopic image based on pathological information guidance according to claim 1, characterized in that: The pathological information of the magnifying endoscope includes four categories, namely, hyperplastic polyps, adenomas, sessile serrated lesions and cancers.

3. The method for generating a diffusion model magnified endoscopic image based on pathological information guidance according to claim 1, characterized in that: The training data set is preprocessed, specifically: Performing loading initialization processing on the training data set, and setting the data loading path, data processing method and balanced sampling method; Read each preset sample source image and target image, perform RGB color space conversion and normalization processing, normalize the pixel value of the source image to the range of [0,1], and normalize the pixel value of the target image to the range of [-1,1], wherein the preset sample includes the source image file name, the target image file name and the pathological information text prompt; According to the text prompt of each preset sample, the sampling weight is calculated, wherein the sampling weight is inversely proportional to the frequency of the text prompt in the data set; The processed image data, text prompt information and corresponding file names are packaged into a dictionary structure and returned for model training.

4. The method for generating a diffusion model magnified endoscopic image based on pathological information guidance according to claim 1, characterized in that: The trained magnifying endoscopic image generation model is tested using the test data set to evaluate the performance of the trained magnifying endoscopic image generation model on unknown data. When the evaluation result is qualified, the final magnifying endoscopic image generation model is obtained, which is specifically: Inputting the background mask image and the corresponding pathological information text prompt in the test data set into the trained magnified endoscopic image generation model to obtain a newly generated magnified endoscopic image; The newly generated magnified endoscopic image is evaluated, and the newly generated magnified endoscopic image is compared with the real image to generate an evaluation result. When the evaluation result reaches a preset result value, the evaluation is qualified, and a final magnified endoscopic image generation model is obtained.

Images