Prostate multi-photon microscopic imaging enhancement method based on structure perception diffusion model
By adopting structure-aware diffusion model and symmetric hyperbolic tangent scheduling mechanism in multiphoton microscopy imaging technology, the problem of high-resolution and clear pathological structural imaging in multiphoton microscopy imaging technology is solved, and efficient denoising and super-resolution enhancement effects are achieved.
Patent Information
- Application Number
- CN202510013029.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-06
- Publication Date
- 2025-05-06
AI Technical Summary
While existing multi-photon microscopy imaging technology achieves high-resolution and clear pathological structure imaging, it is difficult to minimize sample damage at the same time, especially in thick tissue samples of prostate tissue, where noise influence and insufficient high-frequency detail capture are present.
Using a structure-aware diffusion model method, the introduction speed and intensity of high-frequency residuals are optimized by introducing high-frequency residuals during the diffusion process and combining with a symmetric hyperbolic tangent scheduling mechanism to improve the structural clarity and detail retention ability of the image.
It significantly improves the denoising and super-resolution effects of prostate multiphoton microscopy images, effectively retains key pathological structural details, and generates enhanced images that are closer to real high-quality images in data distribution.
Smart Images
Figure CN119941559A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of image processing, and in particular to a prostate multi-photon microscopic imaging enhancement method based on a structure-aware diffusion model. Background Art
[0002] Multiphoton microscopy has become an important technology in the field of biomedical imaging. This technology has the advantages of deep tissue penetration, high-resolution imaging, and minimization of photodamage and photobleaching, and has shown significant advantages in the study of thick tissues and living samples. Therefore, multiphoton microscopy has important application value in the field of medical imaging. Among the signal sources of multiphoton microscopy, combining two-photon excitation fluorescence signals with second harmonic generation signals has become an important complementary method. This complementary method enables multiphoton microscopy to observe cell activities through two-photon excitation fluorescence signals and to display structural organizations through second harmonic generation signals, thus being widely used in imaging cell dynamics, internal structures of thick tissues, and cancer research.
[0003] Despite its many advantages, multiphoton microscopy still faces several technical challenges. These challenges mainly manifest themselves in the trade-offs between image quality, acquisition speed, and sample health. High-resolution imaging usually requires long exposure times and high laser powers, which can improve the signal-to-noise ratio, but are also prone to photodamage and photobleaching. For live cell imaging, these damages are particularly detrimental to sample health. In addition, in thick tissue samples, light scattering and absorption can significantly affect imaging depth. Although the use of longer wavelength lasers can increase penetration depth, it often comes at the expense of spatial resolution and signal-to-noise ratio. Post-processing by generating image stacks through multiple low-power scans is another way to improve the signal-to-noise ratio, but this process is time-consuming and still carries the risk of photodamage when the number of repeated scans is large.
[0004] There are currently a variety of technical means to try to solve the above problems, including image super-resolution and image restoration technology. However, these technologies still have shortcomings in improving the quality of high-resolution pathological images affected by noise, especially when retaining key diagnostic structures. In prostate multiphoton microscopy images, prostate tissue usually presents a flocculent or filamentous structure, showing blurred tissue boundaries. Compared with high-resolution images obtained by repeated scanning and averaging for noise reduction, low-resolution, single-scan images have obvious noise and lack of pathological details. Therefore, in clinical diagnostic applications, achieving high-resolution and clear imaging of pathological structures while minimizing sample damage is still a key problem that needs to be solved. Summary of the invention
[0005] The present invention proposes a prostate multiphoton microscopic imaging enhancement method based on a structure-aware diffusion model, which can achieve prostate multiphoton microscopic image denoising and super-resolution enhancement.
[0006] The present invention adopts the following technical solutions.
[0007] A prostate multiphoton microscopic imaging enhancement method based on a structure-aware diffusion model, which is used for denoising and super-resolution enhancement of prostate multiphoton microscopic images. The method includes a structure-aware diffusion model SADiff and a symmetric hyperbolic tangent scheduling mechanism for effectively controlling high-frequency residuals.
[0008] The structure-aware diffusion model SADiff introduces high-frequency residuals in the diffusion process to enhance the model's ability to capture and retain high-frequency details, so that the generated denoised and super-resolution images are improved in terms of structural clarity and detail retention;
[0009] The symmetric hyperbolic tangent scheduling mechanism ensures that the final state of the diffusion process is close to the distribution of low-quality images through an initially increasing and subsequently decreasing function form, and accurately controls the introduction speed and intensity of high-frequency residuals with the diffusion time step through adjustable hyperparameters to optimize the quality of the enhanced image.
[0010] The method defines image quality as follows: an image with low resolution and low signal-to-noise ratio is defined as an original low-quality image, and an image with high resolution and high signal-to-noise ratio is defined as a high-quality image;
[0011] Assume that the resolutions of the original low-quality image and the high-quality image are H / N×W / N pixels and H×W pixels, respectively; in the structure-aware diffusion model, the original low-quality image is first upsampled by applying a bicubic interpolation algorithm, and the upsampling scaling factor is equal to N, so as to obtain a low-quality image with the same resolution as the high-quality image; the high-quality image is denoted as x0, and the low-quality image is denoted as y0;
[0012] Then given x0 and y0, first normalize the values to [-1, 1], then calculate the offset residual e0 and high-frequency residual h0 by the following formula:
[0013] e0=y0-x0 (1)
[0014] h0=x0-y0 (2).
[0015] The symmetric hyperbolic tangent scheduling mechanism introduces three sequences and They are used to control the introduction speed and intensity of the offset residual, high-frequency residual and Gaussian noise of each diffusion time step t respectively; and The value of t at different time steps is controlled by the exponential scheduling mechanism. The value of is controlled by the symmetric hyperbolic tangent scheduling mechanism;
[0016] The diffusion process gradually transforms the input image x0 into the approximate distribution of the target image y0 through a learnable Gaussian distribution transfer function. This process is defined by the formula as follows:
[0017]
[0018] Among them, x t-1 and x t represents the image of adjacent time steps in the diffusion process; λ and κ are hyperparameters, which control the intensity of high-frequency residual and Gaussian noise introduced in the diffusion process respectively; I represents the unit matrix;
[0019] According to formula (3), we can get x at any time step. t Distribution:
[0020]
[0021] Using the reparameterization technique, we can sample x at any time step by t :
[0022]
[0023] The reverse process is represented by parameter Θ and is defined as follows:
[0024]
[0025] The reverse process starts with Step by step, the latent variable distribution p(x t |y0) is transformed into the target distribution p(x0|y0), which is defined as follows:
[0026]
[0027] According to Bayesian theory, the following results are obtained by calculating formula (3) and formula (7):
[0028]
[0029]
[0030] In the reverse process, the neural network f represented by the parameters Θ is used Θ (·) to predict x0. The structure of the neural network follows the structural design in IDDPM. The final objective function is defined as follows:
[0031]
[0032] The symmetric hyperbolic tangent scheduling mechanism is used to control the sequence The scheduling mechanism defines the parameter max_level to represent the value of θ at different time steps t. t The maximum value of , and the parameters start and end are also set to control the speed and intensity of the introduction of high-frequency residuals over time steps.
[0033] The symmetric hyperbolic tangent scheduling mechanism is described by keeping the noise level low at the initial time step and high θ at the intermediate time steps. t value, so that the high-frequency residual is x t The influence of the distribution mean is more significant to achieve better structure perception.
[0034] The structure-aware diffusion model SADiff realizes structure perception by gradually introducing high-frequency residuals during the diffusion process. It is necessary to control the introduction of high-frequency residuals at each time step.
[0035] In the noise scheduling of the diffusion model, the initial time step maintains a low noise level to facilitate high-quality image generation. When high-frequency residuals are introduced to achieve better structural perception, at the intermediate time step, the high-frequency residuals are used to t The influence of the distribution mean needs to be significant, that is, the intermediate time steps need to maintain a high θ t value; this requirement is met by the hyperbolic tangent function tanh, which is defined as follows:
[0036]
[0037] Symmetric hyperbolic tangent scheduling mechanism is used to control the sequence The method of taking values at different time steps t is as follows: define the parameter max_level, which represents the value of θ in all time steps. t The maximum value of , and set two parameters start and end to flexibly control the introduction speed and intensity of high-frequency residuals over time steps;
[0038] Figure 5 The image of the symmetric hyperbolic tangent scheduling mechanism with different start values is shown. Figure 6 The Python language implementation of the symmetric hyperbolic tangent scheduling mechanism is demonstrated. Figure 7 The images of the diffusion process at different time steps using the hyperbolic tangent scheduling mechanism are shown and compared with the images without introducing high-frequency residuals. When max_level is set to 0.999 and end is set to 2.0, the introduction of high-frequency residuals has a significant impact on the image in the intermediate time steps, that is, from t=7 to t=11, which can highlight the high-frequency information, improve the model's perception of high-frequency details, and achieve structural perception.
[0039] Neural network f used to predict x0 Θ(·), whose model training is as follows Figure 3 As shown, given the total number of time steps T, and the training data set D = {(x0, y0)} N , where N represents the number of data pairs contained in D. The model training process includes the following steps:
[0040] Step S1, randomly sample data pairs (x0, y0) from D;
[0041] Step S2, normalize x0 and y0 to [-1, 1];
[0042] Step S3, calculating the offset residual e0 and the high frequency residual h0 by formula (1) and formula (2);
[0043] Step S4, uniformly sample time steps t from 1 to T;
[0044] Step S5: The random noise ∈ is sampled in
[0045] Step S6: Sample x at time step t using formula (8) t ;
[0046] Step S7: According to the objective function defined by formula (14), the neural network f is optimized using the gradient descent method. Θ (·) parameter θ;
[0047] Step S8: Repeat steps S1 to S7 until the model converges.
[0048] Neural network f used to predict x0 Θ (·), whose model reasoning is as follows Figure 4 As shown, given the total number of time steps T and the low-quality image y0 to be enhanced, the model reasoning process includes the following steps: Step A1, normalize y0 to [-1, 1];
[0049] Step A2: The random noise ∈ is sampled in
[0050] Step A3:
[0051] Step A4, gradually obtain x0 by iteration;
[0052] Step A5: Output the iterative result x0 of the last step.
[0053] In step A4, in each iteration, the following steps are performed:
[0054] ①If t>1, from Sampling random noise ∈, if t = 1, let ∈ = 0;
[0055] ②Use the output of the neural network as x0 and calculate x through the sampling form of formula (11) t-1 .
[0056] Compared with the prior art, the present invention has the following beneficial effects:
[0057] The present invention discloses a method for enhancing multiphoton microscopic imaging of the prostate based on a structure-aware diffusion model. The specific innovations are as follows: the existing available methods have the technical difficulties of insufficient capture and retention of high-frequency details, significant noise impact, insufficient super-resolution performance, and lack of optimization for pathological needs. The present invention proposes a structure-aware diffusion model SADiff, which significantly enhances the model's ability to capture and retain high-frequency details by introducing high-frequency residuals in the diffusion process, so that the generated denoised and super-resolution images can achieve significant improvements in structural clarity and detail retention.
[0058] In addition, the present invention also designs a symmetric hyperbolic tangent scheduling mechanism. This mechanism ensures that the final state of the diffusion process is close to the distribution of low-quality images through the function form of initial increase and subsequent decrease, and at the same time, the introduction speed and intensity of high-frequency residuals with the diffusion time step are accurately controlled through adjustable hyperparameters, thereby optimizing the quality of enhanced images. Through experimental verification, the present invention shows superior performance in prostate multiphoton microscope image denoising and super-resolution enhancement, can effectively retain key pathological structure details, and generate enhanced images that are closer to real high-quality images in data distribution. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0060] Attached Figure 1 Schematic diagram of the model framework of the structure-aware diffusion model SADiff of the present invention;
[0061] Attached Figure 2 A schematic diagram of the network structure used in the present invention;
[0062] Attached Figure 3 It is a pseudo code diagram of the model training process of the structure-aware diffusion model SADiff of the present invention;
[0063] Attached Figure 4 It is a pseudo code diagram of the model reasoning process of the structure-aware diffusion model SADiff of the present invention;
[0064] Attached Figure 5 It is a schematic diagram of the image of the symmetric hyperbolic tangent scheduling mechanism under different values of the present invention;
[0065] Attached Figure 6This is a schematic diagram of the Python language implementation of the symmetric hyperbolic tangent scheduling mechanism of the present invention;
[0066] Attached Figure 7 A schematic diagram showing the comparison of the results of introducing high-frequency residuals and not introducing high-frequency residuals at different time steps in the present invention;
[0067] Attached Figure 8 It is a schematic diagram of the enhancement result of the structure-aware diffusion model SADiff of the present invention on data set 1;
[0068] Attached Fig. 9 Schematic diagram of the enhancement results of the structure-aware diffusion model SADiff of the present invention on dataset 2. DETAILED DESCRIPTION
[0069] As shown in the figure, a prostate multiphoton microscopic imaging enhancement method based on a structure-aware diffusion model is used for denoising and super-resolution enhancement of prostate multiphoton microscopic images. The method includes a structure-aware diffusion model SADiff and a symmetric hyperbolic tangent scheduling mechanism for effectively controlling high-frequency residuals.
[0070] The structure-aware diffusion model SADiff introduces high-frequency residuals in the diffusion process to enhance the model's ability to capture and retain high-frequency details, so that the generated denoised and super-resolution images are improved in terms of structural clarity and detail retention;
[0071] The symmetric hyperbolic tangent scheduling mechanism ensures that the final state of the diffusion process is close to the distribution of low-quality images through an initially increasing and subsequently decreasing function form, and accurately controls the introduction speed and intensity of high-frequency residuals with the diffusion time step through adjustable hyperparameters to optimize the quality of the enhanced image.
[0072] The method defines image quality as follows: an image with low resolution and low signal-to-noise ratio is defined as an original low-quality image, and an image with high resolution and high signal-to-noise ratio is defined as a high-quality image;
[0073] Assume that the resolutions of the original low-quality image and the high-quality image are H / N×W / N pixels and H×W pixels, respectively; in the structure-aware diffusion model, the original low-quality image is first upsampled by applying a bicubic interpolation algorithm, and the upsampling scaling factor is equal to N, so as to obtain a low-quality image with the same resolution as the high-quality image; the high-quality image is denoted as x0, and the low-quality image is denoted as y0;
[0074] Then given x0 and y0, first normalize the values to [-1, 1], then calculate the offset residual e0 and high-frequency residual h0 by the following formula:
[0075] e0=y0-x0 (1)
[0076] h0=x0-y0 (2).
[0077] The symmetric hyperbolic tangent scheduling mechanism introduces three sequences and They are used to control the introduction speed and intensity of the offset residual, high-frequency residual and Gaussian noise of each diffusion time step t respectively; and The value of t at different time steps is controlled by the exponential scheduling mechanism. The value of is controlled by the symmetric hyperbolic tangent scheduling mechanism;
[0078] The diffusion process gradually transforms the input image x0 into the approximate distribution of the target image y0 through a learnable Gaussian distribution transfer function. This process is defined by the formula as follows:
[0079]
[0080] Among them, x t-1 and x t represents the image of adjacent time steps in the diffusion process; λ and κ are hyperparameters, which control the intensity of high-frequency residual and Gaussian noise introduced in the diffusion process respectively; I represents the unit matrix;
[0081] According to formula (3), we can get x at any time step. t Distribution:
[0082]
[0083] Using the reparameterization technique, we can sample x at any time step by t :
[0084]
[0085] The reverse process is represented by parameter Θ and is defined as follows:
[0086]
[0087] The reverse process starts with Step by step, the latent variable distribution p(x t |y0) is transformed into the target distribution p(x0|y0), which is defined as follows:
[0088]
[0089] According to Bayesian theory, the following results are obtained by calculating formula (3) and formula (7):
[0090]
[0091]
[0092] In the reverse process, the neural network f represented by the parameters Θ is used Θ (·) to predict x0. The structure of the neural network follows the structural design in IDDPM. The final objective function is defined as follows:
[0093]
[0094] The symmetric hyperbolic tangent scheduling mechanism is used to control the sequence The scheduling mechanism defines the parameter max_level to represent the value of θ at different time steps t. t The maximum value of , and the parameters start and end are also set to control the speed and intensity of the introduction of high-frequency residuals over time steps.
[0095] The symmetric hyperbolic tangent scheduling mechanism is described by keeping the noise level low at the initial time step and high θ at the intermediate time steps. t value, so that the high-frequency residual is x t The influence of the distribution mean is more significant to achieve better structure perception.
[0096] The structure-aware diffusion model SADiff realizes structure perception by gradually introducing high-frequency residuals during the diffusion process. It is necessary to control the introduction of high-frequency residuals at each time step.
[0097] In the noise scheduling of the diffusion model, the initial time step maintains a low noise level to facilitate high-quality image generation. When high-frequency residuals are introduced to achieve better structural perception, at the intermediate time step, the high-frequency residuals are used to t The influence of the distribution mean needs to be significant, that is, the intermediate time steps need to maintain a high θ t value; this requirement is met by the hyperbolic tangent function tanh, which is defined as follows:
[0098]
[0099] Symmetric hyperbolic tangent scheduling mechanism is used to control the sequence The method of taking values at different time steps t is as follows: define the parameter max_level, which represents the value of θ in all time steps. t The maximum value of , and set two parameters start and end to flexibly control the introduction speed and intensity of high-frequency residuals over time steps;
[0100] Figure 5 The image of the symmetric hyperbolic tangent scheduling mechanism with different start values is shown. Figure 6 The Python language implementation of the symmetric hyperbolic tangent scheduling mechanism is demonstrated. Figure 7The images of the diffusion process at different time steps using the hyperbolic tangent scheduling mechanism are shown and compared with the images without introducing high-frequency residuals. When max_level is set to 0.999 and end is set to 2.0, the introduction of high-frequency residuals has a significant impact on the image in the intermediate time steps, that is, from t=7 to t=11, which can highlight the high-frequency information, improve the model's perception of high-frequency details, and achieve structural perception.
[0101] Neural network f used to predict x0 Θ (·), whose model training is as follows Figure 3 As shown, given the total number of time steps T, and the training data set D = {(x0, y0)} N , where N represents the number of data pairs contained in D. The model training process includes the following steps:
[0102] Step S1, randomly sample data pairs (x0, y0) from D;
[0103] Step S2, normalize x0 and y0 to [-1, 1];
[0104] Step S3, calculating the offset residual e0 and the high frequency residual h0 by formula (1) and formula (2);
[0105] Step S4, uniformly sample time steps t from 1 to T;
[0106] Step S5: The random noise ∈ is sampled in
[0107] Step S6: Sample x at time step t using formula (8) t ;
[0108] Step S7: According to the objective function defined by formula (14), the neural network f is optimized using the gradient descent method. Θ (·) parameter θ;
[0109] Step S8: Repeat steps S1 to S7 until the model converges.
[0110] Neural network f used to predict x0 Θ (·), whose model reasoning is as follows Figure 4 As shown, given the total number of time steps T and the low-quality image y0 to be enhanced, the model reasoning process includes the following steps: Step A1, normalize y0 to [-1, 1];
[0111] Step A2: The random noise ∈ is sampled in
[0112] Step A3:
[0113] Step A4, gradually obtain x0 by iteration;
[0114] Step A5: Output the iterative result x0 of the last step.
[0115] In step A4, in each iteration, the following steps are performed:
[0116] ①If t>1, from Sampling random noise ∈, if t = 1, let ∈ = 0;
[0117] ②Use the output of the neural network as x0 and calculate x through the sampling form of formula (11) t-1 .
[0118] Embodiment 1:
[0119] The technical solution of this embodiment is described in detail below in conjunction with the accompanying drawings.
[0120] This embodiment relates to a method for enhancing multiphoton microscopic imaging of the prostate based on a structure-aware diffusion model, including: proposing a structure-aware diffusion model SADiff, introducing high-frequency residuals in the diffusion process, significantly enhancing the model's ability to capture and retain high-frequency details, and significantly improving the generated denoised and super-resolution images in terms of structural clarity and detail retention; designing a symmetric hyperbolic tangent scheduling mechanism to effectively control the introduction of high-frequency residuals. This mechanism ensures that the final state of the diffusion process is similar to the distribution of low-quality images through an initial increasing and subsequent decreasing function form, and at the same time, accurately controls the introduction speed and intensity of high-frequency residuals with the diffusion time step through adjustable hyperparameters, thereby optimizing the quality of the enhanced image.
[0121] The following is the specific implementation process of this embodiment.
[0122] Figure 1 The model framework of the structure-aware diffusion model SADiff proposed in this embodiment is demonstrated. In this embodiment, a low-resolution, low signal-to-noise ratio image is defined as an original low-quality image, and a high-resolution, high signal-to-noise ratio image is defined as a high-quality image. The resolutions of the original low-quality image and the high-quality image are H / N×W / N pixels and H×W pixels, respectively. In this embodiment, the bicubic difference algorithm is first applied to the original low-quality image for upsampling (scaling factor equals N) to obtain a low-quality image with the same resolution as the high-quality image. The high-quality image is denoted as x0, and the low-quality image is denoted as y0.
[0123] Given x0 and y0, this embodiment first normalizes the values to [-1, 1], and then calculates the offset residual e0 and the high-frequency residual h0 using the following formula:
[0124] e0=y0-x0 (1)
[0125] h0=x0-y0 (2)
[0126] Next, this embodiment introduces three sequences and , which are used to control the introduction speed and intensity of the offset residual, high-frequency residual and Gaussian noise of each diffusion time step t. and The value of t at different time steps is controlled by the exponential scheduling mechanism. The value of is controlled by the symmetric hyperbolic tangent scheduling mechanism. The diffusion process gradually transforms the input image x0 into the approximate distribution of the target image y0 through a learnable Gaussian distribution transfer function. This process is defined as follows:
[0127]
[0128] Among them, x t-1 and x t represents the image of adjacent time steps during the diffusion process. λ and κ are hyperparameters that control the intensity of high-frequency residual and Gaussian noise introduced during the diffusion process, respectively. I represents the unit matrix. According to formula (3), we can get x at any time step: t Distribution:
[0129]
[0130] Using the reparameterization technique, we can sample x at any time step by t :
[0131]
[0132] The reverse process is represented by parameter Θ and is defined as follows:
[0133]
[0134] The reverse process starts with Step by step, the latent variable distribution p(x t |y0) is transformed into the target distribution p(x0|y0), which is defined as follows:
[0135]
[0136] According to Bayesian theory, from formula (3) and formula (7), the following results are calculated:
[0137]
[0138] In the reverse process, a neural network f represented by parameters Θ is used Θ (·) to predict x0, the structure of the neural network is as follows Figure 2As shown, it follows the structural design in IDDPM. The final objective function is defined as follows:
[0139] Figure 3 and Figure 4 The process of model training and model inference is shown respectively. Specifically, for model training, Figure 3 As shown, given the total number of time steps T, and the training data set D = {(x0, y0)} N , where N represents the number of data pairs contained in D. The model training process is as follows: (1) Randomly sample data pairs (x0, y0) from D; (2) Normalize x0 and y0 to [-1, 1]; (3) Calculate the offset residual e0 and high-frequency residual h0 using formulas (1) and (2); (4) Uniformly sample time steps t from 1 to T; (5) The random noise ∈ is sampled in; (6) The x at the sampling time step t is obtained by formula (8) t ; (7) According to the objective function defined in formula (14), the gradient descent method is used to optimize the neural network f Θ (·) parameter θ; (8) Repeat steps (1)-(7) until the model converges. For model inference, Figure 4 As shown in Figure 2, given the total number of time steps T and the low-quality image y0 that needs to be enhanced, the model reasoning process is as follows: (1) normalize y0 to [-1, 1]; (2) Sample random noise ∈; (3) Let (4) Obtain x0 step by step through iteration. In each iteration, perform the following steps: ① If t>l, ② Use the output of the neural network as x0 and calculate x through the sampling form of formula (11) t-1 ; (5) Output the iterative result x0 of the last step.
[0140] The symmetric hyperbolic tangent scheduling mechanism in this example is as follows:
[0141] The structure-aware diffusion model SADiff proposed in this embodiment achieves structure perception by gradually introducing high-frequency residuals during the diffusion process. It is crucial to control the introduction of high-frequency residuals at each time step. In the noise scheduling design of the diffusion model, maintaining a sufficiently low noise level at the initial time step is conducive to high-quality image generation. This principle also applies to the introduction of high-frequency residuals. In order to achieve better structure perception, at the intermediate time step, the high-frequency residuals are used to t The influence of the mean of the distribution needs to be significant. To this end, the intermediate time steps need to maintain a high θ t The hyperbolic tangent (tanh) function can effectively meet this requirement and is defined as follows:
[0142]
[0143] The diffusion process aims to transform x0 into an approximate distribution of y0, which requires that the perturbation of the high-frequency residual to the distribution of x0 first increases and then decreases with the growth of the time step. To this end, this embodiment proposes a symmetric hyperbolic tangent scheduling mechanism for controlling the sequence The scheduling mechanism defines a parameter max_level, which represents the value of θ in all time steps. t The maximum value of , and two parameters start and end are set to flexibly control the introduction speed and intensity of high-frequency residuals over time steps.
[0144] In this embodiment, max_level is set to 0.999 and end is set to 2.0.
[0145] Figure 5 The image of the symmetric hyperbolic tangent scheduling mechanism with different start values is shown. Figure 6 The Python language implementation of the symmetric hyperbolic tangent scheduling mechanism is demonstrated. Figure 7 The images of the diffusion process using the hyperbolic tangent scheduling mechanism at different time steps are shown and compared with the images without the introduction of high-frequency residuals. It can be seen that during the intermediate time steps (t = 7 to t = 11), the introduction of high-frequency residuals has a significant impact on the image, highlighting the high-frequency information, thereby improving the model's perception of high-frequency details and achieving structural perception.
[0146] Embodiment 2:
[0147] This example is experimentally verified based on the above embodiment. The prostate tissue samples used in the experiment are from the tissue microarray provided by the First Affiliated Hospital of Fujian Medical University. A multiphoton microscopy imaging system is used for image acquisition to obtain two sets of data sets. The imaging system specifically includes a Zeiss LSM 880 microscope and a femtosecond titanium sapphire laser, which generates 140fs pulses at a frequency of 80MHz and a wavelength of 810nm. The second harmonic generation signal extraction band is 395-420nm, and the two-photon excitation fluorescence signal extraction band is 430-710nm, both of which are encoded in green. The image is scanned and acquired through the HRZ200 fine-tuning focusing platform and cropped to the required size to construct a data set. The two sets of data sets are named Dataset 1 and Dataset 2, respectively.
[0148] Dataset 1 contains 14,000 unpaired images, which are evenly divided into four categories based on Gleason score annotations. Each category contains 3,500 images, corresponding to benign glands, grade 3, grade 4, and grade 5, respectively. 3,000 images are divided from each category as the training set and 500 images as the validation set. The images in Dataset 1 were acquired through 16 repeated scans and averaged to reduce noise. The cropped resolution is 300×300 pixels. Dataset 2 contains 6,144 pairs of low-quality and high-quality paired images, 5,056 pairs of images are used as the training set, and 1,088 pairs of images are used as the validation set. Dataset 2 is not annotated with Gleason scores. The resolution of low-quality images is 128×128 pixels, which are derived from a single scan and contain significant noise; the resolution of high-quality images is 512×512 pixels, which are obtained through 16 repeated scans and averaged to minimize noise. Due to the mechanical return difference of the fine-tuning focus platform, some image pairs may be slightly misaligned. Additionally, there may be slight differences in brightness between paired images due to differences in signal intensity and photon dwell time.
[0149] The specific evaluation indicators in this example are:
[0150] The experiment used two types of image quality evaluation indicators: reference indicators and non-reference indicators. Reference image quality evaluation indicators include peak signal-to-noise ratio (PSNR) and perceptual similarity (LPIPS), and non-reference image quality evaluation indicators include blind image quality evaluation model (BRISQUE) and natural image quality evaluation model (NIQE). In addition, the experiment also indirectly evaluates the enhanced performance of the model through classification results based on Gleason scores. Classification indicators include precision, recall, and F1-Score. The definitions of these classification indicators are as follows:
[0151]
[0152] Among them, TP represents the number of samples that are actually positive and correctly predicted (true positive), FP represents the number of samples that are predicted to be positive but are actually negative (false positive), and FN represents the number of samples that are actually positive but are predicted to be negative (false negative).
[0153] The model parameters selected in this example are as follows:
[0154] The parameters of the structure-aware diffusion model SADiff proposed in this example need to be predefined, including the following key hyperparameters:
[0155] (1) T: the time step number of the diffusion model, which is fixed to 15 in the present invention.
[0156] (2)λ: Parameter that controls the strength of high-frequency residual.
[0157] (3)κ: parameter that controls the intensity of Gaussian noise.
[0158] (4) start: One of the parameters of the symmetric hyperbolic tangent scheduling mechanism, which controls the sequence Values at different time steps.
[0159] (5)p η : Parameters of the exponential scheduling mechanism, controlling the sequence Values at different time steps.
[0160] (6)p ψ : Parameters of the exponential scheduling mechanism, controlling the sequence Values at different time steps.
[0161] To determine the best parameter combination, this example uses a three-step parameter adjustment process. Step 1: Set κ = 1.0, p η =0.3, p ψ =0.6, adjust the combination of different λ and start, and select the best combination; Step 2: Based on the best λ and start selected in the first step, fix p η = 0.3, adjust κ and p ψ , select the best combination; Step 3: fix the best λ, start, κ and p ψ Based on the optimization of p η The parameter adjustment process is based on data set 2, and the optimal parameter combination is finally determined as: λ = 1.0, start = -4.0, κ = 1.0, p ψ =0.6, p η =0.25. During the parameter adjustment process, the effects of different parameter settings on image quality were comprehensively evaluated by combining quantitative analysis with qualitative comparison. The final parameter combination combines the results of these indicators to ensure the optimized performance of the generated image in terms of clarity, noise level, and structural detail fidelity.
[0162] The qualitative results in this example are as follows:
[0163] Figure 8 and Fig. 9 The enhancement results of the structure-aware diffusion model SADiff proposed in the present invention on dataset 1 and dataset 2 are shown respectively. As can be seen from the figure, SADiff shows excellent performance in both the double super-resolution task and the quadruple super-resolution task under noise conditions, and can generate enhanced images with high resolution and clarity while effectively retaining key pathological structural details.
[0164] The quantitative results of this example are as follows:
[0165] Table 1 Comparison of enhanced performance of different methods on dataset 1 and dataset 2
[0166]
[0167]
[0168] Table 1 shows the comparison results of the enhanced performance of different methods on dataset 1 and dataset 2.
[0169] The structure-aware diffusion model SADiff proposed in this example achieved the best results in LPIPS, BRISQUE and NIQE indicators, significantly outperforming other comparison methods. The results show that the enhanced image generated by SADiff is closer to the original high-quality image. Its enhanced result not only has high definition, but also effectively retains key pathological structural details, which can provide more reliable support for subsequent image analysis and diagnosis.
[0170] Table 2 Comparison of classification performance of different methods on dataset 1
[0171]
[0172] Table 2 shows the classification performance comparison results of different methods on dataset 1. The experiment uses all the images in dataset 1 to train a classification model based on the ResNet-50 network architecture. The experimental steps include: first, the low-resolution image is upsampled by applying the bicubic interpolation algorithm to increase its resolution to 256×256, and the upsampled image is used for model verification, and the obtained results are used as the benchmark performance indicators. Subsequently, the structure-aware diffusion model SADiff proposed in the present invention and a variety of comparison algorithms are used to generate their own enhanced images, which are used for model verification and their performance indicators are compared. The results show that the enhanced images generated by SADiff achieve significant improvements of 0.0968, 0.2160 and 0.2324 in average precision, average recall and average F1 score respectively compared with the benchmark performance, while the performance indicators of other comparison algorithms are lower than the benchmark. This shows that the structure-aware diffusion model SADiff proposed in the present invention can effectively retain key pathological structural details and generate enhanced images that are closer to real high-quality images in data distribution, thereby showing better performance in classification tasks.
[0173] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.
Claims
1. A prostate multiphoton microscopic imaging enhancement method based on a structure-aware diffusion model is used for denoising and super-resolution enhancement of prostate multiphoton microscopic images, characterized by: The method includes a structure-aware diffusion model SADiff, and also includes a symmetric hyperbolic tangent scheduling mechanism for effectively controlling high-frequency residuals; The structure-aware diffusion model SADiff introduces high-frequency residuals in the diffusion process to enhance the model's ability to capture and retain high-frequency details, so that the generated denoised and super-resolution images are improved in terms of structural clarity and detail retention; The symmetric hyperbolic tangent scheduling mechanism ensures that the final state of the diffusion process is close to the distribution of low-quality images through an initially increasing and subsequently decreasing function form, and accurately controls the introduction speed and intensity of high-frequency residuals with the diffusion time step through adjustable hyperparameters to optimize the quality of the enhanced image.
2. The method for enhancing prostate multiphoton microscopic imaging based on a structure-aware diffusion model according to claim 1, characterized in that: The method defines image quality as follows: a low-resolution, low-signal-to-noise ratio image is defined as an original low-quality image, and a high-resolution, high-signal-to-noise ratio image is defined as a high-quality image; the resolutions of the original low-quality image and the high-quality image are assumed to be H / N×W / N pixels and H×W pixels, respectively; in the structure-aware diffusion model, the original low-quality image is first upsampled by applying a bicubic interpolation algorithm, and the upsampling scaling factor is equal to N, so as to obtain a low-quality image with the same resolution as the high-quality image; the high-quality image is denoted as x0, and the low-quality image is denoted as y0; Then given x0 and y0, first normalize the values to [-1, 1], then calculate the offset residual e0 and high-frequency residual h0 by the following formula: e0=y0-x0 (1) h0=x0-y0 (2).
3. The method for enhancing prostate multiphoton microscopic imaging based on a structure-aware diffusion model according to claim 2, characterized in that: The symmetric hyperbolic tangent scheduling mechanism introduces three sequences and They are used to control the introduction speed and intensity of the offset residual, high-frequency residual and Gaussian noise of each diffusion time step t respectively; in, and The value of t at different time steps is controlled by the exponential scheduling mechanism. The value of is controlled by the symmetric hyperbolic tangent scheduling mechanism; The diffusion process gradually transforms the input image x0 into the approximate distribution of the target image y0 through a learnable Gaussian distribution transfer function. This process is defined by the formula as follows: Among them, x t-1 and x t represents the image of adjacent time steps in the diffusion process; λ and κ are hyperparameters, which control the intensity of high-frequency residual and Gaussian noise introduced in the diffusion process respectively; I represents the unit matrix; According to formula (3), we can get x at any time step. t Distribution: Using the reparameterization technique, we can sample x at any time step by t : The reverse process is represented by parameter Θ and is defined as follows: The reverse process starts with Step by step, the latent variable distribution p(x t |y0) is transformed into the target distribution p(x0|y0), which is defined as follows: According to Bayesian theory, the following results are obtained by calculating formula (3) and formula (7): In the reverse process, the neural network f represented by the parameters Θ is used Θ (·) to predict x0. The structure of the neural network follows the structural design in IDDPM. The final objective function is defined as follows:
4. The method for enhancing prostate multiphoton microscopic imaging based on a structure-aware diffusion model according to claim 3, characterized in that: The symmetric hyperbolic tangent scheduling mechanism is used to control the sequence The scheduling mechanism defines the parameter max_level to represent the value of θ at different time steps t. t The maximum value of , and the parameters start and end are also set to control the speed and intensity of the introduction of high-frequency residuals over time steps.
5. The method for enhancing prostate multiphoton microscopic imaging based on a structure-aware diffusion model according to claim 4, characterized in that: The symmetric hyperbolic tangent scheduling mechanism is described by keeping the noise level low at the initial time step and high θ at the intermediate time steps. t value, so that the high-frequency residual is x t The influence of the distribution mean is more significant to achieve better structure perception.
6. The method for enhancing prostate multiphoton microscopic imaging based on a structure-aware diffusion model according to claim 5, characterized in that: The structure-aware diffusion model SADiff realizes structure perception by gradually introducing high-frequency residuals during the diffusion process. It is necessary to control the introduction of high-frequency residuals at each time step. In the noise scheduling of the diffusion model, the initial time step maintains a low noise level to facilitate high-quality image generation. When high-frequency residuals are introduced to achieve better structural perception, at the intermediate time step, the high-frequency residuals are used to t The influence of the distribution mean needs to be significant, that is, the intermediate time steps need to maintain a high θ t value; this requirement is met by the hyperbolic tangent function tanh, which is defined as follows:
7. The method for enhancing prostate multiphoton microscopic imaging based on a structure-aware diffusion model according to claim 6, characterized in that: Symmetric hyperbolic tangent scheduling mechanism is used to control the sequence The method of taking values at different time steps t is as follows: define the parameter max_level, which represents the value of θ in all time steps. t The maximum value of , and two parameters start and end are set to flexibly control the speed and intensity of the introduction of high-frequency residuals over time steps; when max_level is set to 0.999 and end is set to 2.0; during the intermediate time steps, that is, from t = 7 to t = 11, the introduction of high-frequency residuals has a significant impact on the image, which can highlight the high-frequency information, improve the model's perception of high-frequency details, and realize structural perception.
8. The method for enhancing prostate multiphoton microscopic imaging based on a structure-aware diffusion model according to claim 3, characterized in that: Neural network f used to predict x0 Θ (·), when training the model, the total number of time steps T is given, and the training data set D = {(x0, y0))} N , where N represents the number of data pairs contained in D. The model training process includes the following steps: Step S1, randomly sample data pairs (x0, y0) from D; Step S2, normalize x0 and y0 to [-1, 1]; Step S3, calculating the offset residual e0 and the high frequency residual h0 by formula (1) and formula (2); Step S4, uniformly sample time steps t from 1 to T; Step S5: The random noise ∈ is sampled in Step S6: Sample x at time step t using formula (8) t ; Step S7: According to the objective function defined by formula (14), the neural network f is optimized using the gradient descent method. Θ (·) parameter θ; Step S8: Repeat steps S1 to S7 until the model converges.
9. The method for enhancing prostate multiphoton microscopic imaging based on a structure-aware diffusion model according to claim 8, characterized in that: Neural network f used to predict x0 Θ (·), when the model is inferred, given the total number of time steps T and the low-quality image y0 to be enhanced, the model inference process includes the following steps: Step A1, normalize y0 to [-1, 1]; Step A2: The random noise ∈ is sampled in Step A3: Step A4, gradually obtain x0 by iteration; Step A5: Output the iterative result x0 of the last step.
10. The method for enhancing multiphoton microscopic imaging of prostate based on structure-aware diffusion model according to claim 8, characterized in that: In step A4, in each iteration, the following steps are performed: ①If t>1, from Sampling random noise ∈, if t = 1, let ∈ = 0; ②Use the output of the neural network as x0 and calculate x through the sampling form of formula (11) t-1 .