A self-supervised super-resolution reconstruction method for structured light illumination microscopy

Abstract

This invention belongs to the field of image reconstruction technology, specifically relating to a self-supervised super-resolution reconstruction method for structured light illumination microscopy. The method includes: acquiring multiple frames of low-resolution original images containing different orientations and phase modulation information from a structured light microscopy imaging system; preprocessing the images, performing denoising before spectral separation to reduce the impact of noise on subsequent reconstruction, retaining the traditional structured light microscopy physical reconstruction process, using a learnable matrix to model the phase and amplitude parameters of the spectral reconstruction, iteratively optimizing and updating, adaptively estimating the phase and amplitude, and reconstructing a high-resolution image; inputting the high-resolution image into a forward physical imaging model to generate a low-resolution image, constructing a self-supervised loss function with the original image, minimizing the loss to constrain reconstruction, and achieving physically consistent self-supervised optimization. This invention improves reconstruction stability and quality under low signal-to-noise ratio imaging conditions without requiring high-resolution labeled data, and enhances high-frequency detail preservation while ensuring physical consistency.

Description

Technical Field

[0001] This invention belongs to the field of image reconstruction technology, specifically relating to a super-resolution reconstruction method based on self-supervised structured light illumination microscopy. Background Technology

[0002] Structured light illumination microscopy (SIM) is a microscopic imaging technique that uses multiple illumination images with different phases and orientations to perform spectral separation and reconstruction in the frequency domain to obtain super-resolution imaging results. Existing SIM reconstruction methods mainly include frequency domain reconstruction methods based on physical models, such as reconstruction methods based on parameter estimation and Wiener filtering, as well as end-to-end reconstruction methods based on deep learning proposed in recent years.

[0003] Physical model-based methods rely on accurate estimation of parameters such as phase, modulation depth, and frequency shift. Under low signal-to-noise ratio (SNR) imaging conditions, they are susceptible to noise interference, leading to inaccurate parameter estimations and consequently causing spectral aliasing, artifact enhancement, or even reconstruction failure. While deep learning-based methods have improved reconstruction results in some scenarios, they typically require large amounts of high-quality paired labeled data. Furthermore, the reconstruction process lacks constraints on the physical imaging mechanism, easily introducing non-realistic structures and affecting the physical reliability of the imaging results.

[0004] Therefore, existing technologies struggle to simultaneously achieve reconstruction stability, physical consistency, and detail fidelity under low signal-to-noise ratio conditions. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a self-supervised super-resolution reconstruction method for structured light illumination microscopy, comprising:

[0006] S1. Acquire multiple frames of low-resolution raw images acquired by the structured light microscopy system, wherein the low-resolution raw images contain different orientation and phase modulation information;

[0007] S2. Perform denoising preprocessing on the original image to obtain a denoised image;

[0008] S3. Perform a Fourier transform on the denoised image to convert it into a frequency domain image;

[0009] S4. Construct a phase parameter estimation network, extract features from the frequency domain image, and output the phase parameters corresponding to each direction and phase.

[0010] S5. Based on the frequency domain image and the estimated phase parameters, perform spectral separation and Wiener reconstruction to obtain an initial high-resolution reconstructed image;

[0011] S6. Input the initial high-resolution reconstructed image into the structured light forward physical imaging model to generate a simulated low-resolution image;

[0012] S7. Construct a self-supervised loss function that includes a mean squared error term and a structural similarity term, minimize the loss function and introduce high-frequency guiding constraints, and iteratively optimize the phase parameter estimation network and reconstruction parameters.

[0013] S8. After iterative convergence, output the final high-resolution reconstructed image.

[0014] The beneficial effects of this invention are:

[0015] 1) SIM super-resolution reconstruction can be completed without high-resolution labeled data, reducing dependence on experimental data acquisition conditions;

[0016] 2) It can effectively suppress the influence of noise on parameter estimation and spectral separation under low signal-to-noise ratio imaging conditions, thereby improving reconstruction stability;

[0017] 3) Through a self-supervised optimization mechanism constrained by a physical model, the physical consistency of the reconstruction results is guaranteed, and the generation of artifacts is reduced;

[0018] 4) The high-frequency guidance mechanism enhances the detail fidelity of the reconstruction results, which is superior to the traditional Wiener reconstruction method in terms of both objective indicators and visual effects. Attached Figure Description

[0019] Figure 1 This is a diagram of a self-supervised network structure for a super-resolution reconstruction method based on structured light illumination microscopy. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] A self-supervised super-resolution reconstruction method for structured light illumination microscopy includes:

[0022] S1. Acquire multiple frames of low-resolution raw images acquired by the structured light microscopy system, wherein the low-resolution raw images contain different orientation and phase modulation information;

[0023] S2. Perform denoising preprocessing on the original image to obtain a denoised image;

[0024] S3. Perform a Fourier transform on the denoised image to convert it into a frequency domain image;

[0025] S4. Construct a phase parameter estimation network, extract features from the frequency domain image, and output the phase parameters corresponding to each direction and phase.

[0026] S5. Based on the frequency domain image and the estimated phase parameters, perform spectral separation and Wiener reconstruction to obtain an initial high-resolution reconstructed image;

[0027] S6. Input the initial high-resolution reconstructed image into the structured light forward physical imaging model to generate a simulated low-resolution image;

[0028] S7. Construct a self-supervised loss function that includes a mean squared error term and a structural similarity term, minimize the loss function and introduce high-frequency guiding constraints, and iteratively optimize the phase parameter estimation network and reconstruction parameters.

[0029] S8. After iterative convergence, output the final high-resolution reconstructed image.

[0030] The forward physical imaging model of structured illumination microscopy (SIM) can be described as follows: excitation light is spatially modulated to form a sinusoidal fringe illumination pattern that illuminates the sample. The illumination pattern is multiplied by the sample fluorescence distribution, and then convolved by the microscope's optical system to form an image. Finally, a low-resolution image is acquired by the detector. For illumination in the d-th direction and the p-th phase, the acquired low-resolution image... It can be represented as:

[0031]

[0032] Where r represents spatial coordinates, A sinusoidal illumination pattern in the d-th direction and the p-th phase. For sample fluorescence distribution, Let be the system optical point spread function (PSF), ⊗ denotes the convolution operation, and N(r) be additive noise.

[0033] Structured lighting patterns The expression is:

[0034]

[0035] in, The average illumination intensity is m, and the modulation depth is m. Let be the fringe frequency vector in the d-th direction. Let be the phase shift of the p-th phase.

[0036] Performing a Fourier transform on the above imaging process yields the frequency domain imaging model:

[0037]

[0038] Substituting the spectrum of the sinusoidal illumination pattern and expanding it yields a system of equations containing linear combinations of the frequency components of the sample spectrum. Traditional SIM reconstruction relies on estimating the phase. Modulation depth m and frequency shift vector The parameters are used to construct a separation matrix to demix each spectral component, and then Wiener filtering and deconvolution are applied to obtain a super-resolution image. However, under low signal-to-noise ratio conditions, explicit estimation of parameters such as phase and modulation depth is easily affected by noise, leading to inaccurate construction of the spectral separation matrix, causing spectral aliasing and reconstruction artifacts.

[0039] This invention transforms the process of traditional SIM frequency domain reconstruction, which relies on explicit phase estimation, into parameter modeling in the spatial domain based on learnable linear mappings. For example... Figure 1 As shown, specifically, a phase parameter estimation network consisting of three fully connected layers is constructed. Its structure is as follows:

[0040] Input layer: Receives preprocessed low-resolution image feature vectors, with dimensions of... ;

[0041] Hidden layer: Contains a fully connected layer with dimension 1. Followed by the ReLU activation function;

[0042] Output layer: Fully connected layer, output dimension is This corresponds to the number of phase parameters that need to be estimated (such as the phase shift values ​​of P phases in each of the D directions).

[0043] A network can be formally represented as:

[0044]

[0045] in The learnable weight matrix for each layer, For bias vectors, This represents the phase estimate output by the network. The network is trained end-to-end through a subsequent self-supervised optimization process, requiring no real phase-labeled data.

[0046] Before spectral separation, a denoising module D(⋅) is introduced to preprocess the input image for denoising in each direction phase. This denoising module can employ a convolutional denoising network structure based on residual learning, with the original low-resolution image as its input. Output denoised image The denoised image then enters the spectral separation and reconstruction process, thereby reducing the interference of noise on parameter estimation and spectral separation, and improving the reconstruction stability under low signal-to-noise ratio conditions.

[0047] Self-supervised loss function:

[0048] The self-supervised loss function of this invention is constructed based on physical closed-loop constraints: the reconstructed high-resolution image Simulated low-resolution image sequences were regenerated using a forward-looking physical imaging model. and the low-resolution image of the original input. Consistency constraints are applied. The loss function is a weighted combination of the mean squared error term and the structural similarity term:

[0049]

[0050] The mean squared error loss term (MSE) is defined as follows:

[0051]

[0052] Where D: number of structured light illumination directions; P: number of phase steps in each direction; M, N: height and width of the image (number of pixels). (x,y): The pixel value at position (x,y) of the low-resolution image of the d-th direction and p-th phase that was actually acquired; (x,y): The pixel value at position (x,y) of the simulated low-resolution image generated from the reconstructed high-resolution image by the forward physical imaging model;

[0053] Structural similarity loss term Defined as:

[0054]

[0055] in, The structural similarity index between two images is calculated using the following formula:

[0056]

[0057] in, : Original images and reconstructing simulated images The local mean; : These are the local standard deviations of the two images, respectively; Local covariance of two images; : A stability constant used to avoid the denominator being zero, where L is the dynamic range of the pixel value, typically taking values ​​of k1=0.01, k2=0.0; The weighting coefficients used to balance the contributions of the two loss terms satisfy the following conditions: .

[0058] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A super-resolution reconstruction method based on self-supervised structured light illumination microscopy, characterized in that, include: S1. Acquire multiple frames of low-resolution raw images acquired by the structured light microscopy system, wherein the low-resolution raw images contain different orientation and phase modulation information; S2. Perform denoising preprocessing on the original image to obtain a denoised image; S3. Perform a Fourier transform on the denoised image to convert it into a frequency domain image; S4. Construct a phase parameter estimation network, extract features from the frequency domain image, and output the phase parameters corresponding to each direction and phase. S5. Based on the frequency domain image and the estimated phase parameters, perform spectral separation and Wiener reconstruction to obtain an initial high-resolution reconstructed image; S6. Input the initial high-resolution reconstructed image into the structured light forward physical imaging model to generate a simulated low-resolution image; S7. Construct a self-supervised loss function that includes a mean squared error term and a structural similarity term, minimize the loss function and introduce high-frequency guiding constraints, and iteratively optimize the phase parameter estimation network and reconstruction parameters. S8. After iterative convergence, output the final high-resolution reconstructed image.

2. The super-resolution reconstruction method for structured light illumination microscopy based on self-supervised method according to claim 1, characterized in that, The low-resolution original image includes: in, This represents the low-resolution original image in the d-th direction and p-th phase at spatial coordinates r, where r represents the spatial coordinates. This represents a sinusoidal structured lighting pattern in the d-th direction and the p-th phase. Indicates the fluorescence distribution of the sample. The system's optical point spread function (PSF) is represented. This represents the convolution operation, and N(r) represents additive noise.

3. The super-resolution reconstruction method for structured light illumination microscopy based on self-supervised method according to claim 1, characterized in that, The denoising preprocessing employs a convolutional denoising network based on residual learning, with the original low-resolution image as input. Output denoised image .

4. The super-resolution reconstruction method for structured light illumination microscopy based on self-supervised method according to claim 1, characterized in that, The phase parameter estimation network is a three-layer fully connected network, which includes an input layer, a hidden layer and an output layer in sequence. The input layer receives image feature vectors, the hidden layer uses the ReLU activation function, and the output layer outputs a D×P dimensional phase parameter vector. The network is trained end-to-end in a self-supervised manner, without the need for real phase annotation.

5. The super-resolution reconstruction method for structured light illumination microscopy based on self-supervised method according to claim 1, characterized in that, A phase parameter estimation network is used to extract features from the frequency domain image, outputting the phase parameters corresponding to each phase in each direction, including: in, This represents the phase estimate output by the phase parameter estimation network. These represent the learnable weight matrices for the input layer, hidden layer, and output layer, respectively. These represent the bias vectors of the input layer, hidden layer, and output layer, respectively. denoted as the phase parameter estimation network, and x represents the frequency domain image.

6. The super-resolution reconstruction method for structured light illumination microscopy based on self-supervised method according to claim 1, characterized in that, The spectral separation demixes the spectral components of the aliased samples in the frequency domain, and the Wiener reconstruction is used to suppress noise and deconvolve to recover high-resolution information.

7. The super-resolution reconstruction method for structured light illumination microscopy based on self-supervised method according to claim 1, characterized in that, The self-supervised loss function includes: in, Represents the self-supervised loss function. This represents the weights of the mean squared error loss term between the original image and the reconstructed image. This represents the mean squared error loss term. This represents the weights of the structural similarity loss term between the original image and the reconstructed image. This represents the structural similarity loss term.

Images