Super-resolution fluorescence image reconstruction method based on diffusion model and related equipment

Abstract

The invention provides a diffusion model-based super-resolution fluorescence image reconstruction method and related equipment, and the method is suitable for the technical field of structured light illumination microscope super-resolution imaging, and comprises the steps: obtaining a to-be-reconstructed microscopic image; inputting the microscopic image to be reconstructed into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result; and inputting the first preprocessing result into a reverse phase of a pre-trained diffusion model to carry out iterative denoising sampling so as to obtain a reconstructed super-resolution fluorescence image corresponding to the microscopic image to be reconstructed. According to the scheme, image reconstruction is carried out by using a progressive generation mechanism of the diffusion model and a preprocessing result of the residual channel attention network, the problem of neural network frequency spectrum offset is effectively solved, a better detail reduction capability is shown in image reconstruction, and an image reconstruction effect is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of super-resolution imaging of structured light illumination microscopes, and in particular to a super-resolution fluorescence image reconstruction method based on a diffusion model and related equipment. Background Art

[0002] Super-resolution fluorescence microscopy breaks the constraints of optical diffraction limits and provides a powerful observational tool for exploring the physiological activities of nanoscale subcellular structures. However, super-resolution imaging is not "what you see is what you get" and relies heavily on post-processing reconstruction algorithms, which places high demands on both algorithm performance and the quality of the original imaging results.

[0003] Traditional physics-based image reconstruction algorithms mostly require precise optical parameters and time-consuming iterative calculations. In practical applications, they are greatly affected by adverse factors such as noise and aberrations, which greatly limits the observation capabilities of super-resolution microscopes under the requirements of low phototoxicity and photobleaching, resulting in poor image reconstruction effects. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a super-resolution fluorescence image reconstruction method and related devices based on a diffusion model to solve the problems of poor image reconstruction effect in traditional physics-based image reconstruction algorithms.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] A first aspect of an embodiment of the present invention discloses a super-resolution fluorescence image reconstruction method based on a diffusion model, the method comprising:

[0007] Acquiring a microscopic image to be reconstructed;

[0008] Inputting the microscopic image to be reconstructed into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result;

[0009] Inputting the first preprocessing result into the reverse phase of a pre-trained diffusion model for iterative denoising sampling to obtain a reconstructed super-resolution fluorescence image corresponding to the microscopic image to be reconstructed;

[0010] The residual channel attention network and the diffusion model are trained based on a preset semi-synthetic dataset.

[0011] Preferably, the microscopic image to be reconstructed is input into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result, including:

[0012] The microscopic image to be reconstructed is input into a pre-trained residual channel attention network for denoising and prediction to obtain a first preprocessing result, wherein the first preprocessing result at least includes: a first denoised image obtained by denoising and a first point spread function obtained by prediction.

[0013] Preferably, the process of training the residual channel attention network and the diffusion model based on a preset semi-synthetic dataset includes:

[0014] constructing a semi-synthetic dataset covering subcellular structures based on an optical degradation model of a microscopic imaging system, wherein the semi-synthetic dataset includes at least original images with different signal-to-noise ratios and resolutions;

[0015] Using the semi-synthetic dataset and loss function, training the residual channel attention network until convergence;

[0016] Obtaining a second preprocessing result, where the second preprocessing result is obtained by preprocessing the original image in the semi-synthetic dataset using the converged residual channel attention network, the second preprocessing result at least including: a second denoised image obtained by denoising and a second point spread function obtained by predicting;

[0017] The diffusion model is trained using the second preprocessing result until convergence.

[0018] Preferably, a semi-synthetic dataset covering subcellular structures is constructed based on an optical degradation model of a microscopic imaging system, comprising:

[0019] Based on the optical degradation model of the microscopic imaging system, different Gaussian blur kernels and mixed noise are simulated;

[0020] The Gaussian blur kernel, the mixed noise and true multi-SNR widefield images are combined to construct a semi-synthetic dataset covering subcellular structures.

[0021] Preferably, training the diffusion model using the second preprocessing result until convergence includes:

[0022] In the forward phase of the diffusion model, Gaussian noise is gradually added to the sampling target of the diffusion model until the sampling target becomes pure noise, and the sampling target is a true value distribution;

[0023] In the reverse phase of the diffusion model, the second preprocessing result is input into the diffusion model, so that the diffusion model gradually restores the sampling target by predicting the Gaussian noise added in the forward phase, thereby completing the training of the diffusion model.

[0024] A second aspect of an embodiment of the present invention discloses a super-resolution fluorescence image reconstruction device based on a diffusion model, the device comprising:

[0025] an acquisition unit, configured to acquire a microscopic image to be reconstructed;

[0026] A preprocessing unit, configured to input the to-be-reconstructed microscopic image into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result;

[0027] a reconstruction unit, configured to input the first preprocessing result into a reverse phase of a pre-trained diffusion model for iterative denoising sampling, so as to obtain a reconstructed super-resolution fluorescence image corresponding to the microscopic image to be reconstructed;

[0028] The residual channel attention network and the diffusion model are trained by a training unit based on a preset semi-synthetic dataset.

[0029] Preferably, the preprocessing unit is specifically used to: input the microscopic image to be reconstructed into a pre-trained residual channel attention network for denoising and prediction to obtain a first preprocessing result, and the first preprocessing result at least includes: a first denoised image obtained by denoising and a first point spread function obtained by prediction.

[0030] Preferably, the training unit includes:

[0031] A construction module is used to construct a semi-synthetic dataset covering subcellular structures based on an optical degradation model of a microscopic imaging system, wherein the semi-synthetic dataset at least includes original images with different signal-to-noise ratios and resolutions;

[0032] A first training module is configured to train a residual channel attention network using the semi-synthetic dataset and the loss function until convergence;

[0033] an acquisition module, configured to acquire a second preprocessing result, the second preprocessing result being obtained by preprocessing the original image in the semi-synthetic dataset by the converged residual channel attention network, the second preprocessing result at least comprising: a second denoised image obtained by denoising and a second point spread function obtained by predicting;

[0034] The second training module is used to train the diffusion model using the second preprocessing result until convergence.

[0035] A third aspect of an embodiment of the present invention discloses an electronic device, comprising: a processor and a memory, wherein the processor and the memory are connected via a communication bus; wherein the processor is used to call and execute a program stored in the memory; and the memory is used to store a program, wherein the program is used to implement the super-resolution fluorescence image reconstruction method based on the diffusion model disclosed in the first aspect of the embodiment of the present invention.

[0036] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the super-resolution fluorescence image reconstruction method based on the diffusion model disclosed in the first aspect of the embodiment of the present invention is implemented.

[0037] Based on the above-mentioned embodiments of the present invention, a method and related equipment for super-resolution fluorescence image reconstruction based on a diffusion model are provided. The method comprises the following steps: obtaining a microscopic image to be reconstructed; inputting the microscopic image to be reconstructed into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result; and inputting the first preprocessing result into the reverse phase of a pre-trained diffusion model for iterative denoising sampling to obtain a reconstructed super-resolution fluorescence image corresponding to the microscopic image to be reconstructed. This solution utilizes the progressive generation mechanism of the diffusion model and the preprocessing results of the residual channel attention network for image reconstruction, effectively overcoming the problem of spectral bias in neural networks, demonstrating superior detail restoration capabilities in image reconstruction, and improving image reconstruction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 An example diagram of the spectrum bias phenomenon in the traditional end-to-end deep learning reconstruction method provided by an embodiment of the present invention;

[0040] Figure 2 A flowchart of a super-resolution fluorescence image reconstruction method based on a diffusion model provided by an embodiment of the present invention;

[0041] Figure 3 Flowchart for training residual channel attention network and diffusion model provided by embodiments of the present invention;

[0042] Figure 4 A schematic diagram illustrating the principle of a super-resolution fluorescence image reconstruction method based on a diffusion model provided by an embodiment of the present invention;

[0043] Figure 5 A reconstruction effect diagram for a local data set provided by an embodiment of the present invention;

[0044] Figure 6 A reconstruction rendering of an external data set provided by an embodiment of the present invention;

[0045] Figure 7This is a reconstruction effect diagram of the two-color dynamic imaging of living cells provided by an embodiment of the present invention;

[0046] Figure 8 This is a structural block diagram of a super-resolution fluorescence image reconstruction device based on a diffusion model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0048] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0049] Super-resolution fluorescence microscopy breaks the constraints of optical diffraction limits and provides a powerful observational tool for exploring the physiological activities of nanoscale subcellular structures. However, super-resolution imaging is not "what you see is what you get" and relies heavily on post-processing reconstruction algorithms, which places high demands on both algorithm performance and the quality of the original imaging results.

[0050] As the optical principles and hardware systems of various super-resolution fluorescence microscopes mature, reconstruction algorithms have become the primary factor limiting super-resolution imaging performance. Traditional physics-based image reconstruction algorithms, which mostly require precise optical parameters and time-consuming iterative calculations, are significantly affected by undesirable factors such as noise and aberrations in practical applications. This significantly limits the observation capabilities of super-resolution microscopes that require low phototoxicity and photobleaching resistance.

[0051] Research has shown that deep learning has been widely used in super-resolution reconstruction of microscopic images in recent years, showing significantly better robustness, efficiency, and flexibility than traditional methods in low-photon budget and large-volume data reconstruction scenarios. However, most of these current deep learning methods are based on end-to-end supervised learning strategies, such as Figure 1As shown in the example diagram of spectral bias in traditional end-to-end deep learning reconstruction methods, the end-to-end supervised learning strategy inevitably constrains these deep learning methods from the inherent spectral bias of neural networks. The reconstruction of high-frequency detail information is still unsatisfactory, leaving significant room for improvement in the resolution enhancement performance of these methods. Furthermore, these data-driven image reconstruction methods also suffer from extremely limited generalization due to insufficient training data types and quantity, making deep learning-based reconstruction methods generally applicable only to specific local data modalities.

[0052] With the advancement of computer vision technology, the paradigm of super-resolution reconstruction tasks has continued to see new breakthroughs. Generative AI techniques such as generative adversarial networks and diffusion models, when applied to super-resolution reconstruction, have demonstrated significantly superior detail reasoning capabilities compared to previous methods, significantly improving the shortcomings of super-resolution reconstruction, such as insufficient resolution, inconsistent distribution, and easy loss of weak signals. Whether and how super-resolution reconstruction algorithms based on these advanced deep learning models can be applied to fluorescence microscopy has gradually attracted the attention of researchers, becoming a major research hotspot and development path for AI-enabled microscopy.

[0053] Based on this, this scheme proposes a super-resolution fluorescence image reconstruction method and related equipment based on the diffusion model. The progressive generation mechanism of the diffusion model and the preprocessing results of the residual channel attention network are used for image reconstruction, which effectively overcomes the problem of neural network spectrum bias, shows better detail restoration ability in image reconstruction, and improves the image reconstruction effect.

[0054] Furthermore, this scheme aims to achieve high-fidelity and high-robustness super-resolution reconstruction by integrating the probabilistic generation capability of the diffusion model with the adaptive optimization mechanism of self-inspired degradation learning, breaking through the resolution limit of traditional methods, while reducing dependence on the training data modality and improving the applicability of the model in complex scenarios such as low signal-to-noise ratio and dynamic living cell imaging.

[0055] After applying this solution, the following problems can be solved:

[0056] (1) Traditional physics-based algorithms rely on precise optical parameters and complex iterative operations, which are difficult to deal with interference such as noise and aberrations, resulting in low reconstruction efficiency and insufficient robustness;

[0057] (2) Existing deep learning methods have poor reconstruction effects on high-frequency details due to spectral bias, and are limited by the scale and modality of training data, resulting in weak generalization capabilities.

[0058] (3) Generative models (such as GAN) are prone to introducing artifacts or losing weak signals in fluorescence microscopy image reconstruction, and it is difficult to balance detail reasoning and distribution consistency.

[0059] Overall, this solution adaptively filters out noise from the original imaging results and predicts the corresponding point spread function (PSF), guiding the diffusion model to sample reasonable high-resolution reconstruction results. The following describes this solution in detail through various embodiments.

[0060] See also Figure 2 , shows a flow chart of a super-resolution fluorescence image reconstruction method based on a diffusion model provided by an embodiment of the present invention, the super-resolution fluorescence image reconstruction method comprising:

[0061] Step S201: Acquire a microscopic image to be reconstructed.

[0062] In the specific implementation of step S201 , a microscopic image to be reconstructed is obtained. The microscopic image to be reconstructed is a low-quality microscopic image (original imaging result) obtained in an actual imaging application.

[0063] Step S202: Input the microscopic image to be reconstructed into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result.

[0064] In some specific embodiments, before executing step S202, a semi-synthetic dataset is constructed, and the residual channel attention network (denoted as RCAN) and the diffusion model (also called denoising diffusion probability model, denoted as DDPM) are trained using the constructed semi-synthetic dataset. That is, the residual channel attention network and the diffusion model are trained based on the preset semi-synthetic dataset, and then Figure 3 Detailed instructions are given on how to construct a semi-synthetic dataset, how to train RCAN, and how to train DDPM.

[0065] In the specific implementation of step S202, the microscopic image to be reconstructed is input into the pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result, wherein the preprocessing includes denoising processing and PSF prediction processing.

[0066] Specifically, the microscopic image to be reconstructed is input into a pre-trained residual channel attention network (RCAN) for denoising and prediction to obtain a first preprocessing result. The first preprocessing result at least includes: a first denoised image obtained by denoising and a first point spread function (i.e., a first PSF) obtained by prediction.

[0067] That is, the microscopic image to be reconstructed is subjected to RCAN inference to obtain a first denoised image and a first point spread function output by the RCAN.

[0068] Step S203: inputting the first preprocessing result into the reverse phase of the pre-trained diffusion model to perform iterative denoising sampling to obtain a reconstructed super-resolution fluorescence image corresponding to the microscopic image to be reconstructed.

[0069] In the specific implementation of step S203 , the first preprocessing result is input into the reverse stage of the diffusion model for iterative denoising sampling to obtain a reconstructed super-resolution fluorescence image (super-resolution fluorescence image) corresponding to the microscopic image to be reconstructed.

[0070] Specifically, in the reverse stage of the diffusion model (reverse denoising stage), the first denoised image and the first point spread function output by the residual channel attention network (RCAN) are used as conditional inputs, and the Unet in the diffusion model iteratively denoises and samples the distribution of high-resolution data. This "distribution of high-resolution data" is the reconstructed super-resolution fluorescence image corresponding to the microscopic image to be reconstructed.

[0071] It should be noted that in the process of training the diffusion model, Gaussian noise will be gradually added in the forward phase of the diffusion model (which will be discussed later). Figure 3 Detailed description is given in ), based on this, in practical applications, the direction process of the first denoised image and the first point spread function entering the diffusion model allows Unet to iteratively denoise and sample the distribution of the high-resolution data used for training. The specific content of the distribution of high-resolution data is detailed in formula (1).

[0072] (1);

[0073] In formula (1), x t-1 and x t are the results of the previous time step and the current time step in the iterative denoising process, z is the distribution of Gaussian noise characterization sampling, is the noise predicted by the network, x deno is the denoising result of RCAN (such as the first denoised image), p is the PSF prediction result of RCAN (such as the first point spread function), t is the noise addition time step, is the scheduling factor of noise addition (usually set to a decreasing arithmetic progression), T is the total time step of noise addition, is the cumulative multiplication, .

[0074] The first term in formula (1) is equivalent to the mean of the sampling distribution at time t-1, which is predicted by Unet. Equivalent to the standard deviation of the distribution, it can be selected based on the certainty requirements of the reconstruction results.

[0075] It should be noted that the DDPM-based reconstruction paradigm given by formula (1) above naturally constitutes a probabilistic reconstruction, which conveniently models pixel-level reconstruction uncertainty while providing a fluorescence intensity distribution. The varying distribution of multiple sampling results for the same field of view can quickly demonstrate the varying reliability of reconstructions in different regions, helping users identify potential inference errors.

[0076] In an embodiment of the present invention, image reconstruction is performed using the progressive generation mechanism of the diffusion model and the preprocessing results of the residual channel attention network, which effectively overcomes the problem of neural network spectral bias, demonstrates better detail restoration capabilities in image reconstruction, and improves the image reconstruction effect.

[0077] For the above embodiments of the present invention Figure 2 The residual channel attention network and diffusion model involved in step S202 and step S203 are shown in Figure 3 , shows a flowchart of training the residual channel attention network and diffusion model provided by an embodiment of the present invention, including the following steps:

[0078] Step S301: constructing a semi-synthetic dataset covering subcellular structures based on an optical degradation model of a microscopic imaging system.

[0079] In the specific implementation of step S301, based on the optical degradation model of the microscopic imaging system, different Gaussian blur kernels (with adjustable bandwidth and orientation) and mixed noise (e.g., Poisson shot noise + Gaussian readout noise) are simulated; the Gaussian blur kernel, mixed noise, and real multi-signal-to-noise ratio wide-field images (e.g., dynamic sequences of living cells) are combined to construct a semi-synthetic dataset covering subcellular structures, which at least includes original images with different signal-to-noise ratios and resolutions.

[0080] It should be noted that the cell structure includes but is not limited to clathrin-coated pits, endoplasmic reticulum, microtubules, etc.

[0081] A true multi-SNR wide-field image means performing PSF degradation on real noise, that is, using a blur kernel to perform optical simulation degradation.

[0082] In some specific embodiments, the optical degradation model of the microscopic imaging system can be expressed as formula (2).

[0083] (2);

[0084] In formula (2), I is the actual imaging result, I0 is the ideal imaging, P is the PSF of the microscopic imaging system, △p is the aberration such as defocus and coma, and N is the photoelectron noise.

[0085] To fully simulate the degradation models of different systems and optical parameters, this scheme adds slight noise to Gaussian blur kernels with different bandwidths and orientations to cover all possible PSFs of the real system. N is modeled as a mixture of Poisson shot noise and Gaussian readout noise to varying degrees.

[0086] In addition to the simulated data, paired widefield images with varying signal-to-noise ratios, acquired at different excitation light levels, were also retained in the model's training to enhance its ability to learn from real-world degradation. The constructed semi-synthetic dataset includes five subcellular structures of varying structural complexity: clathrin-coated pits, endoplasmic reticulum, microtubules, myosin, and actin filaments.

[0087] Step S302: Using the semi-synthetic dataset and loss function, train the residual channel attention network until convergence.

[0088] It should be noted that the Residual Channel Attention Network (RCAN) is a cascade of multiple residual modules based on convolutional neural networks. Each residual module contains multiple sub-residual modules. To enhance the ability to extract key features, each sub-residual module incorporates a channel attention mechanism to adaptively scale the weights of different channels at different levels of the manifold.

[0089] This solution uses the Residual Channel Attention Network (RCAN) as a preprocessing module. It adaptively extracts key image features by stacking multi-level residual groups, embeds a channel attention module (SE-block) in the residual block, dynamically adjusts the feature channel weights, and enhances the ability to retain high-frequency details.

[0090] In the specific implementation of step S302, the residual channel attention network is trained until convergence using the constructed semi-synthetic dataset and loss function.

[0091] In some specific embodiments, during the process of training the residual channel attention network, the residual channel attention network is trained to denoise the "test original image" (such as the collected structured illumination microscopic image) and predict the corresponding PSF.

[0092] The loss function for supervised learning of the residual channel attention network is detailed in formula (3).

[0093] L=||xy||1+α||pp y ||1(3);

[0094] In formula (3), x and p are the dual-channel outputs of the residual channel attention network, and x and p can represent the denoising result (i.e., denoised image) and the PSF prediction result respectively. yare “noise-free real or simulated wide-field image” and “PSF true value” respectively, α is the weight factor to balance the learning of the two terms, and L is the loss.

[0095] In the practical application of image reconstruction, RCAN preprocessing reduces the risk of photoelectron noise being resolved as artifacts during the reconstruction process. The synchronously predicted PSF characterizes the distance between inputs of different modes and parameters and the target distribution, laying the foundation for high-quality sampling of subsequent diffusion models.

[0096] Step S303: Obtain a second preprocessing result.

[0097] In the process of specifically implementing step S303, a second preprocessing result is obtained, which is obtained by preprocessing the original image in the semi-synthetic dataset by the converged residual channel attention network. The second preprocessing result at least includes: a second denoised image obtained by denoising and a second point spread function obtained by prediction.

[0098] That is, the second preprocessing result is: the original image in the semi-synthetic data set is input into the converged residual channel attention network for preprocessing, and the second denoised image and the second point spread function are output by the residual channel attention network.

[0099] Step S304: using the second preprocessing result to train the diffusion model until convergence.

[0100] In the specific implementation of step S304 , a diffusion model is constructed. In the forward phase of the diffusion model, Gaussian noise is gradually added to the sampling target of the diffusion model until the sampling target becomes pure noise and the sampling target is a true value distribution.

[0101] Specifically, the high-resolution true value distribution is used as the sampling target of the diffusion model (i.e., the target sampling distribution, denoted as x0). As shown in formula (4), Gaussian noise (denoted as ε) is gradually added to x0 in the forward stage of the diffusion model until x0 becomes pure noise.

[0102] (4);

[0103] In formula (4), is the cumulative multiplication, is the scheduling factor of the noise addition (usually set to a decreasing arithmetic progression), T is the total time step of the noise addition, t is the noise addition time step, x t is the result of x0 after adding noise.

[0104] In the reverse phase of the diffusion model, the second preprocessing result is input into the diffusion model, so that the diffusion model gradually restores the sampling target by predicting the Gaussian noise ε added in the forward phase, thereby completing the training of the diffusion model.

[0105] That is to say, in the reverse stage of the diffusion model, based on the second preprocessing result (second denoised image and second point spread function) output by the residual channel attention network, the sampling target is gradually restored by predicting the Gaussian noise ε added in the forward stage.

[0106] Specifically, in the reverse phase of the diffusion model, the second preprocessing result output by the residual channel attention network is used as the conditional input. The generation process is constrained by the cross-attention mechanism to ensure that the reconstruction result conforms to the real physical degradation model. The training objective of the diffusion model is expressed as formula (5).

[0107] (5);

[0108] In formula (5), θ is the noise prediction network of Unet structure, is the noise predicted by the network, t is the noise adding time step, x deno and p are the denoising result of RCAN (such as the second denoised image) and the PSF prediction result (such as the second point spread function), respectively.

[0109] The training of the diffusion model is completed through the above method.

[0110] In general, this scheme synthesizes original images with different signal-to-noise ratios and resolutions based on the classic model of image degradation, and uses real paired wide-field images with high and low signal-to-noise ratios to jointly guide RCAN for supervised learning of denoising and degradation prediction. Subsequently, the diffusion model is trained based on the output of RCAN as a condition to sample reconstruction results that conform to the true value distribution from Gaussian noise.

[0111] To better understand the overall process of this solution, the following processes A1-A4 are used as examples.

[0112] A1. Construct a semi-synthetic data set. The construction process of the semi-synthetic data set is detailed in the above embodiment of the present invention. Figure 3 The content in step S301.

[0113] A2. Train RCAN based on a semi-synthetic dataset. For details on the training process, see the above embodiment of the present invention. Figure 3 The content in step S302.

[0114] A3. Training the diffusion model. For details on the training process, see the above embodiment of the present invention. Figure 3 The content in step S303.

[0115] A4. Inference sampling. For low-quality microscopic images obtained in actual imaging applications, we first input the clean result of RCAN inference denoising (the first denoised image) and obtain the first point spread function. Then, we enter the reverse phase of the diffusion model to allow Unet to iteratively denoise and sample the distribution of the high-resolution data used for training, thereby obtaining the reconstructed super-resolution fluorescence image. The specific content of the distribution of the high-resolution data is detailed in formula (1).

[0116] From the overall perspective, Figure 4 As shown in the schematic diagram of a principle example of a super-resolution fluorescence image reconstruction method based on a diffusion model, the above processes A1-A4 can be divided into a training phase and an inference phase.

[0117] During the training phase, the data required for training includes true values, simulated wide-field images, simulated noisy wide-field images, real noisy wide-field images, and real wide-field images; the RCAN is trained using the data required for training, and the output of the RCAN is used as the input condition of the diffusion model to train the diffusion model.

[0118] In the inference stage, RCAN denoises and predicts the original input, and uses the denoising result and the predicted point spread function as the input conditions of the diffusion model to obtain the super-resolution reconstruction result.

[0119] In order to demonstrate the image reconstruction effect of this solution, the following content is used as an example.

[0120] like Figure 5 As shown in the reconstruction effect diagram for the local dataset, the images in the "Original Input" column are the microscopic images to be reconstructed, the images in the "Super-Resolution Reconstruction" column are the super-resolution fluorescence images reconstructed using this scheme (corresponding to the original images), and the images in the "True Value" column are the real images.

[0121] like Figure 6 As shown in the reconstruction effect diagram for the external data set, the images in the "Original Input" column are the microscopic images to be reconstructed, the "Traditional Structured Light Microscope Reconstruction" are the images reconstructed using existing technology, and the images in the "Super-resolution Reconstruction" column are the super-resolution fluorescence images reconstructed using this scheme.

[0122] like Figure 7 As shown in the reconstruction effect diagram of living cell two-color dynamic imaging, in the living cell two-color dynamic imaging, the image in the "original input" row is the microscopic image to be reconstructed, and the image in the "super-resolution reconstruction" row is the super-resolution fluorescence image reconstructed using this scheme.

[0123] Through the above Figure 5-Figure 7As can be seen from the shown content, this solution shows better detail restoration capability in image reconstruction and has better image reconstruction effect. In detail, this solution has the following beneficial effects:

[0124] 1. This proposal proposes a progressive generation mechanism for the diffusion model combined with PSF constraints, effectively overcoming the spectral bias problem of neural networks and demonstrating superior detail restoration capabilities in the reconstruction of subcellular structures (such as microtubule edges and actin fibers);

[0125] 2. Self-inspired degradation learning uses a semi-synthetic dataset and real data for joint training, enabling RCAN to adaptively filter out mixed noise and predict diverse PSFs, significantly reducing the model's dependence on specific data modalities while maintaining high accuracy in the reconstruction of unknown samples (foreign datasets).

[0126] 3. The combination of physics-based degradation modeling and data-driven generation strategies can be extended to various microscopic imaging modes such as structured light and confocal microscopy, providing a unified framework for cross-platform super-resolution reconstruction.

[0127] Corresponding to the super-resolution fluorescence image reconstruction method based on the diffusion model provided in the above embodiment of the present invention, see Figure 8 The embodiment of the present invention further provides a structural block diagram of a super-resolution fluorescence image reconstruction device based on a diffusion model. The super-resolution fluorescence image reconstruction device includes: an acquisition unit 100, a preprocessing unit 200 and a reconstruction unit 300.

[0128] The acquisition unit 100 is used to acquire the microscopic image to be reconstructed.

[0129] The preprocessing unit 200 is used to input the microscopic image to be reconstructed into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result.

[0130] In a specific implementation, the preprocessing unit 200 is specifically used to: input the microscopic image to be reconstructed into a pre-trained residual channel attention network for denoising and prediction to obtain a first preprocessing result, and the first preprocessing result at least includes: a first denoised image obtained by denoising and a first point spread function obtained by prediction.

[0131] The reconstruction unit 300 is configured to input the first preprocessing result into the reverse phase of a pre-trained diffusion model for iterative denoising sampling to obtain a reconstructed super-resolution fluorescence image corresponding to the microscopic image to be reconstructed.

[0132] Among them, the residual channel attention network and diffusion model are trained by the training unit based on a preset semi-synthetic dataset.

[0133] In an embodiment of the present invention, image reconstruction is performed using the progressive generation mechanism of the diffusion model and the preprocessing results of the residual channel attention network, which effectively overcomes the problem of neural network spectral bias, demonstrates better detail restoration capabilities in image reconstruction, and improves the image reconstruction effect.

[0134] Preferably, based on the content of the above-mentioned super-resolution fluorescence image reconstruction device, the training unit includes a construction module, a first training module, an acquisition module and a second training module. The execution principle of each module is as follows:

[0135] A building module is used to construct a semi-synthetic data set covering subcellular structures based on an optical degradation model of a microscopic imaging system, where the semi-synthetic data set at least includes original images with different signal-to-noise ratios and resolutions.

[0136] This building block is specifically used to: simulate different Gaussian blur kernels and mixed noise based on the optical degradation model of the microscopy imaging system; combine Gaussian blur kernels, mixed noise and real multi-signal-to-noise ratio wide-field images to construct a semi-synthetic dataset covering subcellular structures.

[0137] The first training module is used to train the residual channel attention network until convergence using a semi-synthetic dataset and loss function.

[0138] An acquisition module is used to obtain a second preprocessing result, which is obtained by preprocessing the original image in the semi-synthetic dataset by the converged residual channel attention network. The second preprocessing result at least includes: a second denoised image obtained by denoising and a second point spread function obtained by prediction.

[0139] The second training module is used to train the diffusion model using the second preprocessing result until convergence.

[0140] The second training module is specifically used to: in the forward stage of the diffusion model, gradually add Gaussian noise to the sampling target of the diffusion model until the sampling target becomes pure noise and the sampling target is a true value distribution; in the reverse stage of the diffusion model, input the second preprocessing result into the diffusion model, so that the diffusion model gradually restores the sampling target by predicting the Gaussian noise added in the forward stage, thereby completing the training of the diffusion model.

[0141] Preferably, an embodiment of the present invention further provides an electronic device, comprising: a processor and a memory, the processor and the memory being connected via a communication bus; wherein the processor is used to call and execute a program stored in the memory; the memory is used to store the program, and the program is used to implement the super-resolution fluorescence image reconstruction method based on the diffusion model provided in the above method embodiment.

[0142] Preferably, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the super-resolution fluorescence image reconstruction method based on the diffusion model provided in the above method embodiment is implemented.

[0143] In summary, the embodiments of the present invention provide a super-resolution fluorescence image reconstruction method and related equipment based on a diffusion model, which uses the progressive generation mechanism of the diffusion model and the preprocessing results of the residual channel attention network to perform image reconstruction, effectively overcoming the problem of neural network spectral bias, showing better detail restoration capabilities in image reconstruction, and improving the image reconstruction effect.

[0144] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0145] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0146] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A super-resolution fluorescence image reconstruction method based on a diffusion model, characterized in that: The method comprises: Acquiring a microscopic image to be reconstructed; Inputting the microscopic image to be reconstructed into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result; Inputting the first preprocessing result into the reverse phase of a pre-trained diffusion model for iterative denoising sampling to obtain a reconstructed super-resolution fluorescence image corresponding to the microscopic image to be reconstructed; The residual channel attention network and the diffusion model are trained based on a preset semi-synthetic dataset.

2. The method according to claim 1, characterized in that Inputting the microscopic image to be reconstructed into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result, including: The microscopic image to be reconstructed is input into a pre-trained residual channel attention network for denoising and prediction to obtain a first preprocessing result, wherein the first preprocessing result at least includes: a first denoised image obtained by denoising and a first point spread function obtained by prediction.

3. The method according to claim 1, characterized in that The process of training the residual channel attention network and the diffusion model based on a preset semi-synthetic dataset includes: constructing a semi-synthetic dataset covering subcellular structures based on an optical degradation model of a microscopic imaging system, wherein the semi-synthetic dataset includes at least original images with different signal-to-noise ratios and resolutions; Using the semi-synthetic dataset and loss function, training the residual channel attention network until convergence; Obtaining a second preprocessing result, where the second preprocessing result is obtained by preprocessing the original image in the semi-synthetic dataset by the converged residual channel attention network, and the second preprocessing result at least includes: a second denoised image obtained by denoising and a second point spread function obtained by predicting; The diffusion model is trained using the second preprocessing result until convergence.

4. The method according to claim 3, characterized in that Based on the optical degradation model of the microscopy system, a semi-synthetic dataset covering subcellular structures was constructed, including: Based on the optical degradation model of the microscopic imaging system, different Gaussian blur kernels and mixed noise are simulated; The Gaussian blur kernel, the mixed noise and true multi-SNR widefield images are combined to construct a semi-synthetic dataset covering subcellular structures.

5. The method according to claim 3, characterized in that Training the diffusion model using the second preprocessing result until convergence, comprising: In the forward phase of the diffusion model, Gaussian noise is gradually added to the sampling target of the diffusion model until the sampling target becomes pure noise, and the sampling target is a true value distribution; In the reverse phase of the diffusion model, the second preprocessing result is input into the diffusion model, so that the diffusion model gradually restores the sampling target by predicting the Gaussian noise added in the forward phase, thereby completing the training of the diffusion model.

6. A super-resolution fluorescence image reconstruction device based on a diffusion model, characterized in that: The device comprises: an acquisition unit, configured to acquire a microscopic image to be reconstructed; A preprocessing unit, configured to input the to-be-reconstructed microscopic image into a pre-trained residual channel attention network for preprocessing to obtain a first preprocessing result; a reconstruction unit, configured to input the first preprocessing result into a reverse phase of a pre-trained diffusion model for iterative denoising sampling, so as to obtain a reconstructed super-resolution fluorescence image corresponding to the microscopic image to be reconstructed; The residual channel attention network and the diffusion model are trained by a training unit based on a preset semi-synthetic dataset.

7. The device according to claim 6, characterized in that The preprocessing unit is specifically used to: input the microscopic image to be reconstructed into a pre-trained residual channel attention network for denoising and prediction to obtain a first preprocessing result, wherein the first preprocessing result at least includes: a first denoised image obtained by denoising and a first point spread function obtained by prediction.

8. The device according to claim 6, characterized in that The training unit comprises: A construction module is used to construct a semi-synthetic dataset covering subcellular structures based on an optical degradation model of a microscopic imaging system, wherein the semi-synthetic dataset at least includes original images with different signal-to-noise ratios and resolutions; A first training module is configured to train a residual channel attention network using the semi-synthetic dataset and the loss function until convergence; an acquisition module, configured to acquire a second preprocessing result, the second preprocessing result being obtained by preprocessing the original image in the semi-synthetic dataset by the converged residual channel attention network, the second preprocessing result at least comprising: a second denoised image obtained by denoising and a second point spread function obtained by predicting; The second training module is used to train the diffusion model using the second preprocessing result until convergence.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the processor and the memory are connected via a communication bus; wherein the processor is configured to call and execute a program stored in the memory; The memory is used to store a program, and the program is used to implement the super-resolution fluorescence image reconstruction method based on the diffusion model as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the super-resolution fluorescence image reconstruction method based on the diffusion model according to any one of claims 1 to 5 is implemented.

Images